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FOREWORD 


This  CEET  Initiative  is  our  first  attempt  to  prepare  professors  for  the  Scholarship  of  Teaching.  As 
engineering  and  technology  professors,  they  had  little  background  on  teaching  and  learning  or  on 
educational  research.  With  the  ultimate  goals  of  improving  teaching,  increasing  student  learning,  and 
performing  research  on  teaching  and  learning,  it  was  important  to  construct  a foundation  upon  which 
to  build  knowledge,  understanding,  and  meaning.  When  we  say  “to  increase  student  learning,”  we 
mean  that  we  desire  to  deepen  learners’  understanding  and  to  engage  them  in  creating  meanings  that 
are  purposeful  and  relevant  while  in  our  courses  and  learning  experiences.  As  we  begin,  we  are  fully 
cognizant  that  our  vision  and  mission  will  take  some  time  to  achieve  and  that  data  needed  to  provide 
evidence  we  are  accomplishing  our  goals  will  take  time  to  collect  across  many  courses  and  students. 
However,  we  have  begun.  This  initiative  has  engaged  us  in  critical  reflection,  analysis,  and 
development,  culminating  in  formal  classroom  research.  The  process  of  our  reflective  practice  and  the 
results  are  reported  fully  in  this  college  portfolio. 

Themes  of  thought  are  transparent  across  the  sections  of  our  portfolio  and  across  many  different 
constituencies  regarding  teaching,  student  learning,  and  the  Scholarship  of  Teaching.  The  authors, 
ideals,  thoughts,  perspectives,  research,  studies,  and  philosophical  meanderings  all  arrive  at  similar 
places.  The  voices  of  our  authors  seem  to  be  in  harmony  and  agree  that  students  learn  differently,  that 
they  learn  more  if  they  are  engaged  in  purposeful  and  meaningful  activity  while  learning,  and  that 
students  can  achieve  learning  at  the  upper  levels  of  cognitive  processing  if  knowledge  is  learned  in 
authentic  contexts  (“in  situ”  - requiring  real-world  performances,  structured  as  inquiry,  less  didactic, 
and  discovery  oriented)  and  designed  to  result  in  open-ended  solutions  that  may  differ  across  students 
or  cooperative  learning  groups.  Additionally,  learning  is  social  (not  isolated),  and  students  must 
discover  and  learn  to  hear  their  own  inner  “voices,”  such  that  they  openly  question  and  seek  to 
transform  knowledge  and  make  meaning,  thus  transforming  and  recreating  themselves  and  their  world 
and  re-perceiving  their  relationship  to  the  world.  Our  ultimate  reward  would  be  that  they  become  part 
of  the  generative  process  of  life  with  the  capacity  to  create  their  own  future,  applying  a definition  of 
technology  that  resonates  with  me:  “the  science  of  efficient  action  that  extends  human  capacity  and 
potential.”  Real  learning  is  analogous  in  that  it  extends  human  capacity  and  potential  and  makes  it 
possible  also  to  extend  the  capacity  and  potential  of  cultures. 

Educational  research  is  social  science,  which  is  sometimes  “messy”  and  often  very  unlike  engineering 
research  because  of  the  social  aspect.  The  quality  of  empirical  research  on  teaching  and  learning 
varies,  as  do  its  outcomes,  and  there  are  a great  number  of  more  qualitative  or  anecdotal  studies  that 
are  not  always  as  reliable  but  are  often  worthy  of  consideration.  However,  as  we  searched  to  confirm 
our  direction,  we  found  that  thoughts  and  perspectives  of  a variety  of  current  and  well-known 
researchers,  thinkers,  theorists,  and  practitioners,  as  well  as  those  who  established  the  historical 
foundations  of  teaching  and  learning,  were  generally  in  synchrony.  Therefore,  we  felt  there  were 
themes  of  agreement  transcending  across  the  myriad  of  fields  or  schools  of  thought  relating  to 
teaching  and  learning.  It  is  those  consistent  themes  that  we  used  to  confirm  our  direction  and,  where 
necessary,  to  further  inform  our  direction.  Included,  and  possibly  important  to  note,  are  those  who 
study  the  lives  of  college  students.  Thus,  we  have  chosen  to  describe  the  demographics  of  our  students 
using  qualitative  information  alongside  the  standard  statistical  information.  It  is  interesting  that 
although  the  perspective  comes  from  studying  the  student  (as  a “person”)  and  “their  lives,”  those 
studies  make  a significant  contribution  toward  understanding  how  students  learn  best  and  demonstrate 
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or  provide  evidence  of  their  learning.  Students’  feelings,  attitudes,  experiences  (and  lives  in  general) 
go  far  to  provide  insight  and  implications  for  working  with  them  in  the  classroom.  This  information 
supports  the  teaching  and  learning  themes,  confirming  the  best  theories,  practices,  and  strategies  to 
achieve  the  greatest  learning  potential  of  our  students:  from  the  GIs  and  Boomers  forward  to  Gen 
Xers  and  the  Millennials. 

This  is  the  beginning  of  our  quest  not  only  to  engage  in  the  continuous  improvement,  or  quality  cycle, 
of  teaching  and  learning,  but  to  go  beyond  to  begin  to  formally  study  teaching  and  learning  - the 
Scholarship  of  Teaching.  This  is  the  first  edition  of  our  college  portfolio.  To  some,  it  may  be  far  too 
simple.  Others,  at  our  stage,  may  feel  overwhelmed  by  what  we  have  tried  to  accomplish  in  the  first 
“round”  of  the  initiative.  Many  will  respect  how  we  have  tried  to  draw  from  an  overwhelming  and 
complex  amount  of  research  and  information  on  teaching  and  learning,  scholarship,  students,  learning 
communities,  and  more.  Others  may  feel  that  we  have  begun  too  broadly  and  would  be  more  critical 
of  lack  of  depth.  Some  may  feel  that  the  foundation  we  have  created  is  somewhat  superficial;  yet 
others  may  feel  it  is  good  enough  and  use  it  as  their  foundation. 

Regardless,  we  have  tried  to  create  an  informed  foundation  from  which  to  evolve.  Our  foundation, 
program,  research,  results,  and  products  are  presented  here  for  review.  We  realize  that  in  revealing  all 
aspects  of  our  work,  the  door  for  criticism  is  open  much  wider  than  reporting  from  a more  limited  and 
focused  perspective.  However,  since  we  have  adopted  the  models,  strategies,  processes,  and 
techniques  that  we  hope  to  use  in  our  classrooms,  if  honest  and  committed  to  developing  and  hearing 
our  own  inner  voices  and  realizing  when  they  limit  us,  then  we  honor  those  other  voices  that  may 
confirm  our  direction  and  goals.  We  equally  welcome  those  voices  who  would  challenge  us  to 
critically  reflect  upon  our  directions,  thoughts,  and  decisions,  as  those  voices  stimulate  transformation 
by  helping  us  to  either  admit  the  current  reality  and/or  engage  in  discovery  though  intentional  action. 
Thus,  we  hope  we  are  no  longer  in  a reactive  mode,  but  instead  that  we  have  begun  to  create  a new 
teaching,  learning,  and  research  environment.  In  doing  so,  we  hope  to  strengthen  the  authentic  power 
generated  from  within,  where  critical  reflection,  thought,  change,  and  the  community  engagement  of 
different  voices  make  us  all  more  than  we  were  before,  and  that,  individually  and  collectively,  we 
recreate  ourselves. 

We  encourage  you  to  use  our  work,  reflect  upon  it  critically,  identify  its  strengths  and  weaknesses, 
and  ultimately  engage  with  us  in  the  transformation  of  our  teaching  and  learning  environment  and 
new  research  adventure  - the  Scholarship  of  Teaching. 


Jule  Dee  Scarborough,  Ph.D. 
Distinguished  Professor 


Promod  Vohra,  Ed.D.  PE,  Dean 

College  of  Engineering  & Engineering  Technology 
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writing  from  the  author.  However,  those  interested  are  encouraged  to  contact  the  author  as  she  will  approve  most 
requests,  especially  for  faculty  development  purposes.  She  will  also  provide  “live”  versions  of  the  documents  and 
presentations  upon  request. 

Section  B.  This  section  includes  all  data  and  reports.  Source  each  according  to  authorship. 

However,  each  faculty  member  artifact  or  example  is  named.  Therefore,  if  sourcing  an  example 
within  the  section,  source  the  faculty  member  as  well  as  the  section  author.  Source  them  individually 
for  any  artifact  or  program  product  example  used  alone.  Finally,  a DRAFT  manuscript  about  the 
experimental  classroom  research  or  the  implementation  of  new  teaching  and  learning  strategies, 
models,  and  procedures  is  also  included  by  each  faculty  member  participant.  Two  of  these 
manuscripts  have  been  accepted  for  publication;  several  others  are  in  process,  and  the  others  are  going 
to  be  submitted  to  appropriate  engineering  and  technology  journals.  Please  source  the  individual 
authors  for  their  work  respectively,  as  drafts;  some  of  these  are  still  in  a “report”  type  mode,  but  are 
being  changed  into  formal  manuscripts.  These  manuscripts  had  to  be  included  as  they  are  one 
variable  in  the  program  research  and  evaluation  design. 
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Portfolio  Notes  for  USE  or  REPLICATION 


1 . This  Portfolio  contains 

a.  Complete  Faculty  Development  Program:  Although  the  program  was  developed  to  use 
with  engineering  and  technology  faculty,  the  teaching,  student  assessment,  and 
learning  program  components  as  well  as  the  educational  research  components  are 
“generic”  and  can  be  used  without  change  with  any  discipline. 

b.  Instruments,  Feedback  and  Evaluation  Forms,  Program  Worksheets:  These,  as  well, 
can  be  used  without  change  with  any  discipline. 

c.  Research  Designs,  Methodology,  and  Procedures:  For  (a)  the  faculty  development 
program  research  and  evaluation,  (b)  the  classroom  implementation  of  teaching 
models,  assessments,  etc.  and  (c)  the  experimental  classroom  research  on  student 
learning,  the  designs,  methodologies,  procedures,  and  process  for  research  and 
evaluation  can  be  used,  without  change,  with  any  discipline. 

d.  All  data  and  reports  are  transparent.  The  organization,  strategies,  reporting  style, 
process,  and  procedures  will  work  with  any  discipline. 

e.  Presentations:  These  will  also  work  with  any  discipline. 

f.  Faculty  Products:  Throughout  each  section  in  Data  and  Reports  (B.1-14)  faculty 
artifacts  are  included.  Sometimes  there  are  examples  of  all  faculty  products  related  to 
the  program  component;  sometimes  there  are  several,  rather  than  everyone’s.  The 
intent  was  to  reflect  the  quality  of  work,  the  diverse  styles  and  responses  to 
development,  and  the  range  of  quality  - although  that  did  not  vary  greatly;  they  all 
performed  well. 

2.  In  all  of  our  work  on  teaching  and  learning,  regardless  of  whether  secondary  or  post- 
secondary, we  are  firmly  committed  to  the  role  that  general  education  plays  as  a critical 
foundation.  Of  course,  everyone  would  probably  acknowledge  that  point  and  believe  that 
learning  in  the  majors  extends  and  deepens  the  learning  of  those  general  education  disciplines. 
We  further  believe  that  learning  in  the  majors  is  merely  an  extension  and  deepening  of  the 
learning  of  the  general  education  disciplines  within  the  major  context.  This  is  not  to  dismiss 
that  the  major  disciplines  have  their  own  bodies  of  knowledge,  but  the  two  are  so  deeply 
integrated  that  they  cannot  truly  be  separated.  Therefore,  you  will  find,  whether  in  our  work 
with  secondary  schools  or  post-  secondary  instructors  and  professors,  that  we  overtly  address 
the  contributing  general  education  disciplines  equally  alongside  the  engineering  and 
technology  content.  In  our  secondary  work  with  teachers,  you  will  note  on  the  website  below 
that  we  included  English  teachers  as  full  partners.  Math  and  science  was  a natural  inclusion 
when  focusing  on  engineering  and  technology,  but  English,  history,  social  science,  and  other 
disciplines  have  often  been  partners  in  our  math,  science,  and  technology  interdisciplinary 
teams. 

Eor  the  work  herein,  we  have  made  it  very  clear  that  in  any  effort  to  improve  undergraduate 
education  in  the  majors,  one  must  overtly  consider  and  attend  to  the  continued  and  deepened 
learning  regarding  general  education  disciplines,  as  they  are  the  fully  integrated  foundation  for 
any  undergraduate  major.  You  will  see  that  revealed  in  our  process  for  analyzing  and 
rebuilding  or  developing  courses  and  student  assessments. 
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3.  In  several  places  throughout  the  Portfolio,  for  reader  convenience,  partial  pieces  of  other 
sections  are  copied  and  inserted.  This  eases  reader  frustrations  because  when  considering  a 
topic,  everything  is  available  in  that  particular  section.  For  example,  teaching  models  are 
addressed  both  in  Course  Analysis  and  its  section  on  GAPS  Analysis.  However,  Teaching 
Models  is  a major  topic  and  section  on  its  own.  Therefore,  we  copied  the  related  materials 
from  the  Course  Analysis  (B.5)  and  its  subsection  (GAPS  Analysis,  p.21)  into  the  Models  of 
Teaching  Section  (B.l  1),  which  has  other  data  and  reports  included.  This  also  occurs  with 
Test  Analysis  and  Development.  As  with  Models  of  Teaching,  it  was  an  aspect  of  the  earlier 
program  component.  Course  Analysis  (B.6),  but  also  an  equal  program  component  and  was 
completed  in  the  Student  Assessment  Program  component  (B.9).  Therefore,  that  report  was 
copied  into  B.9  as  a subsection. 

4.  Teaching  and  learning  across  all  levels  of  education  are  similar.  If  you  are  interested  in  the 
work  we  have  done  with  secondary  math,  science,  technology,  and  English  interdisciplinary 
teams,  see  strategicalliance.niu.edu.  This  website/electronic  book  is  a similar  type  of 
document  as  this  one.  It  provides  all  program  content  and  information  for  a 21  day  program 
with  secondary  teachers,  the  research  and  evaluation,  and  full  literature,  etc.  (Scarborough, 
2004,  American  Association  of  Higher  Education).  AAHE  has  disbanded;  however,  the  full 
CD  is  available  for  download  at  the  above  website. 

5.  Einally,  when  reading  the  program  description  (A. 3)  or  Data  and  Reports  (C.  1-14),  if  there  are 
any  instruments,  forms,  or  worksheets  that  could  be  useful.  Section  C.1-21  makes  them 
available  in  full  format. 
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INTRODUCTION 
Jule  Dee  Scarborough,  Ph.D. 


This  Portfolio  is  nontraditional  in  that  it  combines  a learning  paper  approach  with  a 
literature  study,  the  presentation  of  two  interdependent  research  endeavors,  design  and 
methodology,  results,  full  program  content  and  process,  evaluation,  and  our  conclusions  , 
recommendations,  and  many  product  samples.  We  are  presenting  a selected  cross  section 
of  the  literature  on  or  related  to  the  scholarship  of  teaching  for  our  faculty  research 
participants,  a more  efficient  and  less  redundant  way  of  moving  faculty  forward  in  a 
timely  manner  than  if  we  expected  each  person  to  produce  his/her  own  version  of  a 
literature  study.  We  are  also  using  this  method  to  present  what  is  most  illuminating, 
informing,  or  relevant  to  our  interests. 

The  research  and  development  aspects  have  two  foci:  (1)  The  Effectiveness  of  a New 
Faculty  Development  Model  and  Program  on  Teaching  and  Student  Learning  in 
Preparing  Faculty  for  the  Scholarship  of  Teaching  and  (2)  Experimental  Classroom 
Research  on  Teaching  and  Student  Learning  as  The  Scholarship  of  Teaching. 

Background 

The  College  of  Engineering  and  Engineering  Technology  (CELT)  at  Northern  Illinois 
University  (NIU)  consists  of  four  departments:  three  engineering  and  one  technology.  It 
has  five  accredited  programs,  either  by  the  American  Board  for  Engineering  and 
Technology  (ABET)  or  the  National  Association  for  Industrial  Technology  (NAIT).  The 
programs  are  electrical  engineering,  industrial  engineering,  mechanical  engineering, 
electrical  and  manufacturing  engineering  technology,  and  industrial  technology.  The 
College  transformed  from  a very  large  and  multifaceted  department  of  industrial 
technology,  with  a proud  history  of  teacher  education,  into  the  newly  structured  college 
in  1985.  The  Department  of  Technology  had  already  accomplished  its  first  new  era  when 
its  programs  evolved  beyond  teacher  education  to  meet  the  needs  of  industry  with  two 
degrees,  education  and  technology,  13  areas  of  emphasis,  and  approximately  1500 
students. 

The  first  years  of  the  second  new  era,  beginning  in  1985,  were  spent  recruiting  faculty; 
designing,  redesigning,  and  developing  programs;  developing  laboratories;  modifying  the 
interim  building;  designing  and  acquiring  funds  for  a new  building;  and  of  course, 
becoming  accredited.  Approximately  five  years  later,  we  moved  from  the  interim 
building  into  a new  building,  while  maintaining  technology  programs  in  the  historical 
building.  The  second  stage  of  that  era,  of  becoming  a more  research  and  development- 
oriented  college,  has  now  begun.  Although  research  and  development  have  been  ongoing, 
new  goals  have  been  established  for  the  college,  and  stronger  administrative  leadership 
and  support  for  faculty  to  develop  and  engage  in  research  and  development  is  now  in 
place.  In  addition  to  seeking  funding  for  research  projects  from  the  well  known  and 
traditional  sources,  such  as  the  National  Science  Foundation,  the  college  has  prioritized 
and  moved  forward  successfully  through  research  and  development  partnership  contracts 
with  business  and  industry. 
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The  Department  of  Technology  continued  to  have  a teacher  education  degree  until  1995, 
and  even  though  the  degree  was  terminated  at  that  time,  the  department  has  sustained  a 
strong  commitment  to  working  with  schools  to  strengthen  mathematics  and  science 
education,  with  an  external  funding  record  of  over  nine  million  dollars  in  grants  from 
federal  and  state  agencies  as  well  as  significant  private  industry  contributions.  Although 
the  focus  of  the  K-12/14  was  primarily  secondary  teaching  and  student  learning,  that 
history,  record,  and  Boyer’s  (1990)  national  call  to  action  for  the  scholarship  of  teaching 
at  the  university  level  lead  us  now  to  internalize  all  that  we  have  researched  and  learned 
about  teaching  and  student  learning  at  the  secondary  level  into  a local  call  for  action  by 
the  college,  one  that  will  focus  on  research  and  development  of  teaching  and  learning  at 
the  higher  education  level  in  engineering  and  technology.  Our  endeavors  will  be 
informed  by  and  reflect  the  philosophy  and  definitions  offered  by  Boyer,  as  well  as 
Glassick,  Huber,  and  Maeroff  (1997)  and  Braxton  (1996).  Boyer  also  informs  us,  beyond 
our  present  consideration  of  the  scholarship  of  teaching,  about  the  other  types  of 
scholarship  where  faculty  potential  can  be  realized.  He  fully  defines  each  one.  His  work 
helps  us  to  reconsider  our  research  goals  and  what  the  whole  schema  for  research  in  the 
college  might  include.  Many  in  higher  education  are  familiar  with  the  ideals  and  thoughts 
of  these  authors,  and  of  the  authors  to  be  mentioned  throughout  the  entire  portfolio. 
Therefore,  as  we  are  beginning  a new  college  adventure  into  the  scholarship  of  teaching, 
while  continuing  to  build  and  expand  research  in  the  traditional  areas  and  in 
developmental  partnerships  with  industry,  it  is  important  for  us  to  inform  our  immediate 
participants  and  to  present  our  vision,  mission,  and  goals.  While  implementing  our  own 
ideas,  we  respectfully  acknowledge  all  the  ideals,  thoughts,  and  work  of  others  who 
inform  or  confirm  our  directions  and  interests. 

Leadership 

In  an  effort  to  model  peer  leadership,  recognize  the  significance  of  the  secondary 
education  successes  on  teaching  and  learning,  and  positively  exploit  our  own  expertise, 
the  dean  of  the  College  supported  a college  peer  and  distinguished  professor  as  the 
primary  leader  of  this  initiative.  As  the  leader  of  the  College,  the  dean  is  vested  in 
moving  the  faculty  roles  and  functions  to  a broader  arena,  and  fundamentally  believes 
that  teaching  and  research  are  equal  and  interdependent  and  that  the  college  cannot 
accomplish  its  mission  without  equal  attention  to  both  teaching  and  research.  Therefore, 
the  Scholarship  of  Teaching  is  a natural  endeavor  for  the  college  philosophically  as  it 
extends  and  deepens  the  teaching  focus,  also  extending  research  into  another  area  and 
leading  us  to  implement  Boyer’s  (1990)  research  model  as  the  college  model.  The  dean 
is  committed  to  leading  change  in  our  bylaws,  if  necessary,  that  reflect  the  acceptance  of 
the  Scholarship  of  Teaching  as  an  area  of  research  to  be  as  equally  valued  for  promotion 
and  tenure  purposes  as  ongoing  discipline-based  research,  pure  or  applied.  It  is  entirely 
possible  that,  because  of  our  historical  acceptance  of  K-14  research  towards  promotion 
and  tenure,  confirmation  of  its  equality  will  be  all  that  is  needed.  Therefore,  this  initiative 
was  conceived,  theoretically  and  conceptually,  and  directed  by  one  of  our  own  faculty 
members  (Burns,  1978). 
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The  CEET  Initiative  on  Teaching  and  Eearning  (CITE)  is  based  upon  the  following: 
CEET  Vision:  To  build  a regional  and  national  reputation  for  The  Scholarship  of 
Teaching. 

CEET  Mission:  To  build  (and  continue  to  expand)  an  interdisciplinary  team  of  faculty 
who  understand  the  four  types  of  scholarship  as  defined  by  Boyer  (1990);  have  the 
ability  to  engage  in  either  or  several  of  the  types  of  scholarship;  and  are  stimulated  to 
engage  in  scholarship  activities,  especially  research  on  teaching  and  student  learning,  in 
their  disciplinary  classrooms. 

CEET  Promotion  and  Tenure  Objective:  To  review  what  constitutes  acceptable 
scholarship  across  its  departments.  To  confirm  that  research  productivity  across  the  types 
of  scholarship,  as  defined  by  Boyer  (1990)  and  including  The  Scholarship  of  Teaching,  is 
as  equally  acceptable  for  promotion  and  tenure  purposes  as  the  research  of  discovery, 
integration,  and  application. 

CEET  Goal  for  Eaculty:  To  engage  in  scholarship  of  teaching  either  as,  or  alongside  of 
other,  scholarship  interest(s). 

CEET  Scholarship  Quality  Goals:  To  adopt  standards  for  quality  performance  in 
scholarship  (Glassick  et  ah,  1997).  This  goal  will  not  be  addressed  in  the  pilot  initiative. 

CEET  Eaculty  Development  Goal:  To  design,  develop,  pilot,  institutionalize,  and  sustain 
a program  of  faculty  development  on  teaching,  student  learning,  student  assessment,  and 
educational  research  to  prepare  faculty  to  engage  in  scholarship  of  teaching  through 
experimental  and/or  action  research  in  the  classrooms. 

CEET  Goal  for  Students:  To  develop  learners  in  the  truest  sense,  meaning  that  students 
leave  with  such  excitement  about  what  they  have  learned  that  they  continue  to  seek  to 
learn,  extending  what  they  learned  with  us  into  new  meanings  in  their  new  experiences 
and  opportunities  throughout  their  careers.  We  want  our  students  to  understand  the 
philosophy  that  learning  stimulates  learning;  each  original  lesson  continues  to  grow, 
change,  expand,  and  take  on  different,  broader,  and  deeper  meanings  as  they  navigate 
through  various  career,  personal,  and/or  additional  learning  events  or  contexts. 

Eaculty  Participants 

A group  of  three  teams,  two  faculty  members  from  each  of  three  departments,  and  one 
faculty  member  from  the  fourth  department  (lost  one  faculty  member)  working  together 
engaged  in  professional  development  on  student  learning,  teaching,  student  assessment, 
and  educational  research.  The  initiative  focused  on  two  parallel,  but  inseparable,  research 
aspects  - the  development  of  faculty  leaders  on  teaching  and  learning  followed  by 
classroom  research  on  the  effectiveness  of  selected  teaching  and  assessment  strategies  on 
student  learning.  Once  the  faculty  members  completed  the  professional  development 
program,  they  continued  into  the  classroom  to  engage  in  formal  experimental  research  to 
evaluate  the  effectiveness  of  selected  strategies.  This  experimental  treatment  data  was 
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compared  to  control  group  data  collected  prior  to  the  professional  development  and 
classroom  research.  The  research  methods  were  both  quantitative  (experimental)  and 
qualitative,  as  student  achievement  and  other  types  of  data  were  collected  throughout  the 
initiative.  In  addition,  the  faculty  development  program  and  model  were  evaluated. 
Therefore,  the  research  was  comprised  of  two  primary  foci:  faculty  development  and  the 
effect  of  selected  teaching  and  assessment  strategies  on  student  learning  - our  first 
attempt  to  execute  the  Scholarship  of  Teaching  through  classroom  research  at  the 
university  level  in  our  college. 

Beginning  very  focused,  the  faculty  development  program  engaged  faculty  in  18  days  of 
the  following  modules:  (1)  cognitive  learning,  models,  and  processes;  (2)  connecting 
national  standards  to  course  and  student  learning  objectives,  outcomes,  and  student 
assessment;  (3)  evidence  of  learning;  improving  traditional  testing  and  measurement,  and 
continuing  on  to  connect  tests  to  performance  assessment  - the  development  of 
performance  tasks  and  corresponding  rubrics;  (4)  teaching  models;  and  (5)  educational 
research.  This  program,  content,  and  process  is  explained  in  detail  in  Section  A. 5. 

Pilot  Initiative  Outcomes 

(also  see  Faculty  Development  Program  Outcomes  in  Section  A. 5) 

1 . Faculty  development  model,  program,  and  process; 

2.  Research  and  evaluation  of  the  faculty  development  model,  program,  and 
process; 

3.  The  Scholarship  of  teaching  through  experimental  classroom  research; 

4.  Classroom  research  results  to  guide  teaching  and  learning  decisions  and  base 
further  research  upon; 

5.  The  redefinition  of  the  relationship  of  teaching  to  research  with  the 
confirmation  of  (or  if  needed,  the  adoption  of  modified)  bylaws  that  will 
reflect  the  four  areas  of  scholarship  as  those  proposed  by  Boyer  (1990): 
discovery,  integration,  application,  and  teaching. 

Products 

1 . CEET  faculty  professional  development  model,  program,  and  outcomes 
portfolio 

2.  Eaculty  leadership  team(s) 

3.  Classroom  or  student  learning  research  results 

4.  New  or  revised  Teaching/Eeaming  educational  products  (e.g.,  student 
assessments,  syllabi,  models  and  processes,  feedback  instruments,  and  others) 

5.  College  Portfolio  to  be  submitted  to  the  ERIC 

6.  National  mailing  to  all  engineering  and  technology  schools  with  portfolio 
enclosed 

7.  Research  Manuscripts  to  submit  to  national  publications 

8.  National  presentations 

9.  Proposals  for  ongoing  research 
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10.  Proposals  to  modify  faculty  evaluation  procedures  to  include  scholarship  of 
teaching  as  an  acceptable  form  of  research  towards  promotion  and  tenure;  this 
will  not  occur  until  the  second  stage  of  the  Initiative. 

Phases  and  Timelines 

Preparation  - January  - December  2005  - Background  research  and  preparation 

Program  and  Research 

1 . February  2006  - May  2006  - Pilot  professional  development  model  and 
program 

2.  August  2006  - December  2006  - Research  on  teaching  and  learning  strategies 
in  classrooms;  conduct  experimental  research 

3.  January  - February  2007  - Evaluation;  data  collection,  processing,  analysis, 
interpretation;  publication  preparation 

4.  2007  - 2008  - Dissemination 

5.  2007-  2008  - Submit  changes  in  the  bylaws  related  to  scholarship  (if  needed) 
or  acknowledge  formal  confirmation  that  The  Scholarship  of  Teaching  is  as 
equally  accepted  for  promotion  and  tenure  as  other  types  of  research. 

Sustainability 

1 . Modify  faculty  development  model,  program,  and  process  based  upon 
evaluation  results 

2.  Modify  courses  based  upon  research  and  feedback  of  2006  research  semester 

3.  Redesign  classroom  research  based  upon  research  results 

4.  Engage  in  round  two  of  program  and  research 

5.  Institutionalize  and  sustain  faculty  development  program  and  faculty 
classrooms  research  with  annual  evaluations  and  updates 

Operational  Objectives 


Eaculty  development 

1 . To  prepare  faculty  to  engage  in  the  Scholarship  of  Teaching  and  its  inherent 
research  agenda 

2.  To  prepare  a faculty  team  to  lead  the  Scholarship  of  Teaching  in  the  college 

3.  To  engage  faculty  in  a program  of  professional  development  on  teaching  and 
learning  and  educational  research 

Research 

1.  To  evaluate  the  effectiveness  of  the  professional  development  program 

2.  To  study  the  effects  of  selected  teaching  and  learning  strategies  on  student 
learning  through  formal  experimental  classroom  research 

3.  To  study  the  effects  of  improved  traditional  testing  in  conjunction  with 
performance  assessment 
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Quality  Improvement 

1 . To  strengthen  teaching  and  student  learning  across  CEET 

2.  To  extend  research  beyond  traditionally  accepted  definitions  of  research  for 
engineering  and  technology  professors  in  the  college,  adding  the  Scholarship 
of  Teaching 

3.  To  build  confidence  in  the  value  and  role  of  teaching  as  a primary  and 
welcome  responsibility  of  our  professoriate; 

4.  To  build  confidence  in  the  value  and  role  of  research  on  teaching  and  learning 
- the  Scholarship  of  Teaching  - and  include  it  equally  alongside  the  other 
types  of  research  across  the  college. 


Evaluation 

1 . Eaculty  development  model,  program  content,  and  processes 

2.  Classroom  research  results 

3.  Outcomes 

4.  Educational  products 

Dissemination  - nationally  through  American  Society  for  Engineering  Education  (ASEE) 
& NAIT  conferences  and  journals;  EE,  IE,  ME,  IT  journals  and  Portfolio 

1 . Eaculty  development  model,  program  content,  and  processes 

2.  Classroom  research  results 

3.  Evaluation  of  faculty  development  program 

4.  Eaculty  improvement  results 

CEET  Promotion  and  Tenure  Policy  and  Procedures 

1.  To  confirm  that  the  Scholarship  of  Teaching  is  acceptable  research  for 
promotion  and  tenure  or,  if  necessary,  submit  bylaw  content  changes  to 
include  the  Scholarship  of  Teaching 

2.  To  make  faculty  aware  of  confirmation  of  acceptance  of  the  SoT,  or 

3.  To  implement  new  bylaws 

The  Relationship  between  Research  and  Teaching 

The  seminal  work  by  Boyer  (1990)  led  the  Carnegie  Eoundation  to  engage  in  the  Ernest 
E.  Boyer  Project  of  the  Carnegie  Eoundation  for  the  Advancement  of  Teaching.  Boyer 
raised  the  issue  of  how  faculty  spend  their  time  and  what  they  are  rewarded  for  doing. 
This  led  to  the  question:  "what  activities  of  the  professoriate  are  most  highly  prized. . . 
[noting]  that  it  is  futile  to  talk  about  improving  the  quality  of  teaching  if,  in  the  end, 
faculties  are  not  given  recognition  for  the  time  they  spend  with  students?"  (p.  xi).  He 
traces  the  debate  throughout  history,  illuminating  the  transitions  and  shifting  priorities  of 
American  higher  education,  noting  that  students  are  often  the  losers  and  further  noting 
that  students 

are  assured  that  teaching  is  important,  that  a spirit  of  community  pervades  the 
campus,  and  that  general  education  is  the  core  of  the  undergraduate 
experience. . . .but  the  reality  is  that,  on  far  too  many  campuses,  teaching  is  not 
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well  rewarded,  and  faculty  who  spend  too  much  time  [working  with  students] 
may  diminish  their  prospects  for  tenure  and  promotion,  (p.  xii) 

Boyer’s  (1990)  goal  in  this  work  is  to  "break  out  of  the  tired  old  debate  and  define,  in 
more  creative  ways,  what  it  means  to  be  a scholar. . .recognize  the  full  range  of  faculty 
talent  and  the  great  diversity  of  functions  higher  education  must  perform...  [stating  that] 
for  American  higher  education  to  remain  vital  we  urgently  need  a more  creative  view  of 
the  work  of  the  professoriate"  (p.  xii).  Most  important  in  his  introduction  is  his 
acknowledgement  of  the  "need  [for]  a climate  in  which  colleges  and  universities  are  less 
imitative,  taking  pride  in  their  uniqueness. . .to  end  the  suffocating  practice  in  which 
[they]  measure  themselves. . .by  external  status  rather  than  by  values  determined  by  their 
own  distinctive  mission"  (p.  xiii).  He  frames  the  question  of  better  education  in  the 
context  of  how  scholarship  is  defined  and  rewarded,  trying  to  reflect  what  he  and  others 
consider  the  full  range  of  academic  and  civic  mandates,  and  describes  four  views  of 
scholarship  - "discovery,  integration,  application,  and  teaching,"  defining  them  as  follows 
(pp.  xii-xiii). 

(1)  Knowledge  for  knowledge  sake  - the  creation  of  a bank  of  knowledge  or  information, 
ready  to  draw  upon  when  the  time  for  intelligence  use  arrives.  (Thomas,  1977) 

Scholarship  of  Discovery  comes  closest  to  what  academics  [identify]  as 
research. . .the  freedom  of  inquiry  and  to  follow,  in  a disciplined  fashion,  an 
investigation  wherever  it  may  lead. . . .Not  just  the  outcomes,  but  the  process,  and 
especially  the  passion,  give  meaning  to  the  effort. 

[Boyer  quotes]  Bowen  (1986),  scholarly  research  reflects  our  pressing, 
irrepressible  need  as  human  beings  to  confront  the  unknown  and  to  seek 
understanding  for  its  own  sake. . .tied  inextricably  to  the  freedom  to  think  freshly, 
to  see  propositions  of  every  kind  in  the  ever  changing  light.  And  it  celebrates  the 
special  exhilaration  that  comes  from  a new  idea  (p.  17). . . . [T]he  probing  mind  of 
the  research  is  an  incalculably  vital  asset  to  the  academy  and  the  world. . .the  very 
heart  of  academic  life. . .the  pursuit  of  knowledge  must  be  assiduously  cultivated 
and  defended. . .the  discovery  of  new  knowledge  is  absolutely  critical,  (pp.  1 17- 
18) 

(2)  Authenticating  knowledge  through  analysis  and  interpretation,  establishing  meaning 
or  original  research  through  interdisciplinary  consideration  and  synthesis. 

Scholarship  of  Integration  - the  need  for  scholars  who  give  meaning  to  isolated 
facts,  putting  them  in  perspective. . .making  connections  across  the  disciplines. . . 
serious,  disciplined  work  that  seeks  to  interpret,  draw  together,  and  bring  new 
insight  to  bear  on  original  research. 

[Boyer  quotes]  Van  Doren,  “[t]he  connectedness  of  things  is  what  the  educator 
contemplates  to  the  limit  of  his  [her]  capacity.”  It  is  through  connectedness  that 
research  ultimately  is  made  authentic. . .closely  related  to  discovery. . .where  fields 
converge. . . [where  one  fits]  one's  own  research  - or  the  research  of  others  - into 
larger  intellectual  patterns . . . .Those  engaged  in  discovery  ask  "What  is  to  be 
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known,  what  is  yet  to  be  found?"  Those  engaged  in  integration  ask,  "What  do  the 
findings  mean!  and  provide  a . . .more  comprehensive  understanding. . .requiring 
critical  analysis  and  interpretation,  (pp.  18-21) 

(3)  Where  scholarship  connects  theory  and  practice  and  proves  its  worth  to  the  nation 

and  world. 

Scholarship  of  Application  - How  can  the  knowledge  be  responsibly  applied  to 
consequential  problems?  How  can  it  be  helpful  to  individuals  as  well  as 
institutions?  Can  social  problems  themselves  define  an  agenda. . .serving  the 
interests  of  the  larger  community. 

[When  considering  the  following  international  perspective  by  Harper  (1906)]  . . . 
Scholarship. . .was  regarded  by  the  British  as  "a  means  and  measure  of  self- 
development," by  the  Germans  as  "an  end  in  itself,  "but  by  Americans  as 
"equipment  for  service."  Self-serving  though  it  [the  American  perspective]  may 
have  been,  this  analysis  had  more  than  a grain  of  truth. . .the  gap  between  the 
academy  and  the  needs  of  the  larger  world. . .service  is  routinely  praised,  but 
accorded  little  attention. 

Colleges  and  universities  have  rejected  service  as  serious  scholarship,  partly 
because  its  meaning  is  so  vague  and  often  disconnected  from  serious  intellectual 
work. . . [e.g.  projects,  committees,  etc.] . Clearly  a distinction  must  be  drawn 
between  citizenship  activities  and  projects  that  relate  to  scholarship  itself. . . .To  be 
considered  scholarship,  service  activities  must  be  tied  directly  to  one's 
professional  activity . . . serious,  demanding  work,  requiring  rigor. . . [and] 
accountability. . .associated  with  research  activities. . .The  process  we  have  in  mind 
is  more  dynamic  [where]  new  intellectual  understandings  arise  out  of  the  very  act 
of  application  [where]  theory  and  practice  vitally  interact,  and  one  renews  the 
other. . . both  apply[ing]  and  contribut[ing]  to  human  knowledge. . . [using]  the 
skills  and  insights  only  the  academy  can  provide. . . 

Handlin  observed  our  troubled  planet  "can  no  longer  afford  the  luxury  of  pursuits 
confined  to  an  ivory  tower”. . . [where]  scholarship  has  to  prove  its  worth  not  on  its 
own  terms  but  by  service  to  the  nation  and  the  world,  (pp.  21-23) 

Scholarship  of  Teaching  - [where]  the  work  of  the  professor  becomes 
consequential. . . .as  it  is  understood  by  others . . .Today  teaching  is  often  viewed  as 
a function. . . [however,]  Aristotle  said,  "Teaching  is  the  highest  form  of 
understanding. "....beginning  with  what  the  teacher  knows. . .steeped  in  the 
knowledge  of  their  fields . . . .One  reason  why  legislators  fail  to  understand  why  10- 
12  hours  in  the  classroom  each  week  can  be  a heavy  load  is  their  lack  of 
awareness  of  the  hard  work  and  serious  study  that  undergirds  good  teaching,. . .a 
dynamic  endeavor  involving  all  analogies,  metaphors,  and  images  that  build 
bridges  between  the  teacher's  understanding  and  the  student's  learning. 
Pedagogical  [and  adrogogical]  procedures  must  be  carefully  planned  and 
continuously  examined. . . [According  to]  Palmer  (1983). . .knowing  and  learning 
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are  communal  acts.  With  this  vision,  great  teachers  create  a common  ground  of 
intellectual  commitment.  They  stimulate  active,  not  passive,  learning  and 
encourage  students  to  be  critical,  creative  thinkers,  with  the  capacity  to  go  on 
learning  after  their  college  days  are  over.  Further,  good  teaching  means  that 
faculty,  as  scholars,  are  also  learners.... [not]  transmit[ting]  information  that 
students  are  expected  to  memorize  and  then  recall. . .but  transforming  and 
extending  it  as  well. . .Inspired  teaching  keeps  the  flame  of  scholarship  alive. . .All 
academics  credit  good  teachers. . .defining  their  work  so  compellingly  that  it 
became. . .a  lifetime  challenge.  Without  the  teaching  function,  the  continuity  of 
knowledge  will  be  broken  and  the  store  of  human  knowledge  dangerously 
diminished. 

Oppenheimer  (1954)  spoke... "The  specialization  of  science  is  an  inevitable 
accompaniment  of  progress;  yet  it  is  full  of  dangers,  and  it  is  cruelly  waster,  since 
so  much  that  is  beautiful  and  enlightening  is  cut  off  from  the  rest  of  the  world. 
Thus,  it  is  proper  to  the  role  of  the  scientist  that  he  [she]  not  merely  find  the  truth 
and  communicate  it  to  his  [her]  fellows,  but  that  he[she]  teach,  that  he  [she]  try  to 
bring  the  most  honest  and  most  intelligible  account  of  new  knowledge  to  all  who 
will  try  to  learn. . .knowledge  is  acquired  through  research,  synthesis,  practice,  and 
through  teaching,  (as  cited  in  Boyer,  1990,  pp.  23-24) 

These  four  types  of  scholarship  acknowledge  the  great  range  of  talent  and  diversity 
within  the  professoriate.  The  creative  tension  between  the  above  definitions  stimulates  us 
to  appreciate  scholarship  from  a broader  perspective,  each  type  contributing  significantly 
to  the  other  and  ultimately  to  the  development  of  humanity  and  its  endeavors  through  the 
academy  and  other  contributing  institutions. 

Also  important  to  us  is  Boyer's  (1990)  point  that  there  is  a unique  opportunity  for 
comprehensive  universities,  one  where  we  can  establish  our  own  unique  missions  rather 
than  imitate  the  traditional  research  universities,  the  opportunity  to  blend  quality  with 
innovation,  choosing  the  foci  of  our  passion  (whether  Scholarship  of  Discovery, 
Scholarship  of  Integration,  Scholarship  of  Application,  or  the  Scholarship  of  Teaching) 
with  the  understanding  that  the  Scholarship  of  Teaching  is  a requirement  for  knowledge 
to  continue  to  expand  and  be  used.  We  agree  with  Boyer  that  "diversity  with  dignity"  (p. 
64)  is  building  a diverse  learning  system  and  learning  organization  where  undergraduate 
and  graduate  learning  are  priorities;  where  the  Scholarship  of  Teaching  is  honored  and 
prioritized  in  conjunction  with  other  types  of  scholarship;  where  they  integrate,  one  not 
dominating  the  other;  and  where  the  diverse  range  is  sought,  sustained,  and  respected. 
Thus,  not  only  is  it  mandatory  that  we  move  ahead  to  formally  acknowledge  the 
importance  of  the  Scholarship  of  Teaching  alongside  the  others,  it  also  reaffirms  and 
acknowledges  the  importance  of  the  historical  commitment  to  teaching  and  student 
learning  in  our  service  to  secondary  education  and  extends  that  work  into  our  own 
engineering  and  technology  classrooms  at  the  university  level  in  an  informed  manner, 
with  an  understanding  of  what  our  vision  means,  what  will  be  required  to  attain  it,  and 
that  the  extension  of  knowledge  rewards  what  will  ensue.  This  vision  attends  to  preparing 
students  for  their  professions  as  learning  individuals,  integrating  general  and  major 
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education  more  relevantly  and  overtly,  strengthening  capstone  experiences,  and 
ultimately  examining  the  master's  degree  educational  experiences  to  determine  what 
needs  to  be  sustained  or  strengthened.  Most  important  is  to  establish  a more  productive 
relationship  between  teaching  and  student  learning.  Therefore,  we  engage  in  this 
initiative  to  actively  explore  that  relationship  through  teaching  and  learning  research.  We 
will  begin  simply  with  a two-pronged,  but  interdependent,  research  initiative:  the  first 
research  focus  will  be  the  design,  development,  and  piloting  of  a new  faculty 
development  model  and  program  to  prepare  a pilot  group  of  faculty  to  actively  engage  in 
experimental  research  on  classroom  teaching  and  student  learning;  the  second  research 
focus  will  be  the  actual  experimental  classroom  research  on  teaching  and  student  learning 
(TL).  Both  of  these  activities  will  be  rigorously  evaluated  to  determine  their  value  to 
inform  the  faculty  and  administration  about  what  the  next  level  of  activities  should  be  to 
sustain,  expand,  and  deepen  the  Scholarship  of  Teaching  initiative.  As  Boyer  (1990) 
establishes,  we  have  the  opportunity  to  determine  our  own  unique  model  and  what  is 
acceptable  as  faculty  role,  function,  and  responsibility.  See  Research  Model  and  Faculty 
Development  Model  A5. 

Wankat,  Felder,  Smith,  and  Oreovicz  (as  cited  in  Huber  & Morreale,  2002)  review  the 
history  of  engineering  research  and  education,  showing  the  progression  toward  research 
on  engineering  education.  They  note  that  until  the  1980s,  it  was  a “we  tried  it  and  liked  it 
and  so  did  the  students”  approach  (p.  217).  When  the  National  Science  Foundation  (NSF) 
began  to  fund  educational  research  and  development,  more  scholarship  began  to  emerge. 
But  the  most  significant  catalyst  may  have  been  ABET’s  Engineering  Criteria  2000.  The 
criteria  required  educational  objectives,  assessment  measures,  and  the  “closing  of  the 
loop”  in  which  assessment  results  were  tied  to  corrective  actions  where  necessary. 
Although  these  new  criteria  do  not  require  educational  research,  they  did  intensify 
interest  in  engineering  education,  assessment,  and  research.  Faculty  members  are 
gradually  realizing  that  changes  in  pedagogy  are  required  if  they  are  to  achieve  the 
outcomes  specified  by  the  criteria.  More  papers  are  being  submitted  on  the  scholarship  of 
teaching  to  the  engineering  education  journals  because  of  the  influence  of  the  criteria. 

Historically,  only  one  of  Boyer’s  (1990)  scholarships  counted  for  faculty  in  the  research 
area,  that  of  discovery.  However,  NSF  fostered  collaboration  across  disciplines  by 
recognizing  that  complex  problems  need  interdisciplinary  attention  for  meaningful 
solutions,  which  required  a shift  from  single  investigator  to  multidisciplinary  centers. 

This  helped  to  make  the  Scholarships  of  Integration  and  Application  more  legitimate  for 
engineering.  But  the  Scholarship  of  Teaching  was  still  not  of  priority,  and  only  a small 
number  of  faculty  participated  in  ASEE;  nor  did  the  few  papers  on  educational  research 
reach  many  faculty  in  the  mainstream.  To  stimulate  more  interest  and  research,  NSE  then 
began  to  support  scholarship  in  engineering  education  through  the  Division  of 
Undergraduate  Education  and  the  Coalitions  program.  This  improved  the  status  of 
educational  research  in  engineering  faculty’s  performance  reviews  and  began  to  move 
collaboration  forward  between  engineering  and  social  science  professors. 

At  the  heart  of  the  matter,  also  mentioned  in  the  Wankat  et  al.  (as  cited  in  Huber  & 
Morreale,  2002)  article,  is  engineering  colleges  and  schools  are  realizing  that  engineering 
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(and  in  our  case  engineering  technology  and  technology)  faculty  must  be  prepared  to 
implement  the  teaching  and  assessment  methodologies  required  to  meet  the  new 
accreditation  standards;  thus,  campuses  are  instituting  faculty  development  programs  and 
initiating  faculty  learning  communities.  However,  there  are  challenges  with  the 
Scholarship  of  Teaching  (and  student  learning)  in  engineering.  Engineering  research  is 
much  more  scientific  and  developmental,  whereas  educational  research  is  much  less 
precise,  as  it  is  social  science  research.  Controlling  the  experimental  design  is  not  as 
easily  accomplished  in  a more  social  scenario. 

The  goal  is  to  improve  teaching  and  learning,  but  who  can  agree  on  what  that  means? 
What  is  learning:  the  acquisition  of  knowledge,  deepening  of  understanding, 
improvement  of  both  technical  and  interpersonal  skills,  development  of  desired  attitudes 
and  values  - all  subjective  constructs.  Defining  them  precisely  is  not  that  easy,  especially 
in  the  engineering  forms  most  commonly  understood  by  engineers.  Educational  research 
cannot  always  be  directly  observed  or  calculated.  Existence  and  development  must 
sometimes  be  inferred  from  observation  of  students’  behaviors,  thus  identification  of  the 
behaviors  and  rules  of  inference  are  controversial.  Students  are  more  difficult  to 
categorize  than  transistors  or  fruit  flies;  the  factors  influencing  learning  (learning  styles, 
personalities,  knowledge  and  skills,  what  they  bring  with  them  (e.g.,  traits  or  home 
environments)  are  all  somewhat  subjective).  Therefore,  to  establish  a cause  and  effect 
relationship  between  a treatment  and  the  outcome  is  difficult  to  demonstrate  or  replicate; 
to  prove  something,  many  studies  have  to  be  executed  with  large  populations  for 
generalization  to  be  possible. 

Engineers  are  not  used  to  this  level  of  involvement  to  determine  a treatment  effect.  The 
reasoning  is  quite  different,  which  builds  skepticism  that  must  be  overcome  to 
accomplish  the  scholarship  of  teaching  and  learning  with  engineers.  Einally,  Wankat  et 
al.  (as  cited  in  Huber  & Morreale,  2002)  note  that  a metric  (tools  and  procedures)  has  to 
be  created  to  determine  a value  for  system  variables.  In  science  and  engineering  proper, 
metrics  and  valid  and  reliable  instruments  to  measure  them  are  easier  to  identify  than  in 
education,  which  has  been  an  obstacle  for  engineering  education  until  recently. 
Engineering  professors  are  becoming  aware  of  teaching  methods,  classroom  assessment 
techniques,  and  although  still  small,  a growing  number  of  engineering  professors  are 
engaging  in  formal  research  studies  of  different  approaches  to  course  design  and  delivery. 
They  are  still  using  mostly  student  surveys  and  end  of  course  ratings  because  they  are 
easy.  These  methods,  however,  lack  credibility  to  achieve  change  with  engineering 
professors  on  teaching  and  learning. 

Although  there  are  published  experimental  studies,  they  have  their  own  set  of  obstacles; 
engineering  classes  are  small,  making  it  difficult  to  have  the  numbers  for  both 
experimental  and  control  groups  and  to  achieve  statistically  significant  results.  We, 
however,  do  not  always  agree  with  that  premise.  Engineering  professors  are  unfamiliar 
with  the  complexities  and  ethical  issues  when  involved  with  human  subjects  and  with  the 
planning  that  has  to  be  accomplished  prior  to  the  studies.  It  seems  that  innovations  in 
engineering  education  usually  develop  through  a natural  growth  and  change  process,  not 
as  often  from  pre-planning.  Therefore,  most  of  the  studies  reported  in  the  Journal  of 
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Engineering  Education  are  rigorous  in  their  quantitative  methods.  Some  have 
methodological  weaknesses,  perhaps  with  the  exception  of  research  on  cooperative 
learning  (Johnson,  Johnson,  & Smith  1998a,  1998b,  1998c;  Springer,  Stanne,  & 

Donovan,  1999).  Qualitative  research  is  making  its  way  into  engineering  education, 
mostly  about  retention,  but  as  faculty  members  realize  that  some  of  the  ABET  2000 
criteria  relate  to  skills  that  can  be  assessed  through  qualitative  methods,  that  will 
gradually  change.  When  assessing  educational  scholarship  in  engineering,  most 
engineering  professors  feel  they  know  high  quality  research.  But  when  considering  the 
Scholarship  of  Teaching  and  Learning,  they  are  not  sure  that  it  can  be  evaluated  with  as 
much  rigor;  learning  that  it  can  will  be  essential  in  making  it  acceptable  criteria  for 
advancing  up  the  engineering  faculty  ladder  for  tenure  and  promotion.  Felder  (2000) 
presents  three  primary  questions  to  use  when  considering  the  promotion  of  a professor  in 
engineering: 

1.  To  what  extent  does  the  instructor’s  teaching  qualify  as  a scholarly  activity? 

2.  How  effective  is  the  instructor’s  teaching? 

3.  How  numerous  and  effective  are  the  instructor’s  educational  research  and 
development  efforts?  (pp.  230-31) 

Felder  (2000)  goes  on  to  suggest  the  data  to  use  when  evaluating  these  questions: 

1 . Archival  data:  lists  of  courses  developed  and  taught,  instructional  materials, 
student  products,  student  numbers,  etc. 

2.  Learning  outcomes  assessment  data:  test  results,  evaluations,  student  self- 
assessments,  etc. 

3.  Subjective  evaluations  by  others 

4.  Self-assessment  data:  teaching  philosophy,  goals,  progress,  etc. 

He  offers  an  educational  scholarship  assessment  protocol.  If  we  are  to  legitimize  the 
Scholarship  of  Teaching  and  Learning  in  the  academic  system  for  engineering  educators, 
then  we  must  integrate  it  into  the  reward  system.  However,  this  is  an  issue  when  the 
financial  support  for  the  Scholarship  of  Teaching  and  Learning  is  weak,  making  it 
difficult  to  get  funding.  And  we  all  know  what  grant  funding  does  for  us!  (In  1999,  72 
papers  were  published  in  the  Journal  of  Engineering  Education:  65%  had  no  funding; 

19%  had  some  funding  from  NSF;  8%  were  funded  by  professors’  universities;  and  3% 
by  the  Federal  Fund  for  the  Improvement  of  Postsecondary  Education.)  Thus,  the  funding 
for  engineering  scholarship  on  teaching  and  learning  is  somewhat  minimal.  There  is  an 
essential  need  for  multidisciplinary  collaboration  with  non-engineers  if  we  are  to  achieve 
a professional  level  of  activity.  For  example,  the  Journal  of  Engineering  Education  cited 
many  non-engineers,  and  NSF  has  funded  most  major  grants  with  co-principal 
investigators  who  are  not  engineers.  This  is  positive,  however,  for  engineers  are  not 
trained  in  educational  research,  ethnographic  methods,  the  construction  of  qualitative 
measures;  they  do  not  have  the  interpretative  skill  or  experience  with  the  broader  range  of 
assessment  methods.  Collaboration  will  bring  more  appropriate  knowledge,  skill,  and 
background  to  the  Scholarship  of  Teaching  and  learning  in  education.  And  engineers 
have  a great  deal  to  offer  their  non-engineering  counterparts  - their  understanding  of 
instructional  technology  for  example.  But  multidisciplinary  collaboration  and  research 
have  its  difficulties  as  well.  Their  “differentness”  is  obstacle  enough  sometimes:  their 
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concept  of  research,  their  priorities,  vocabularies,  and  ability  to  be  team  members  on 
research.  However,  the  partnerships  between  engineers  and  non-engineers/technologists 
and  rewards  can  be  great.  Thus,  the  purpose  and  reasoning  underlying  our  initiative. 

Figure  A.  1.1:  CEET  Scholarship  of  Teaching  Eearning  Community  Model 

Global  Communities  of  Practice  and  Interest 


Regional  & National  Public  Communities  of  Interest 


Business  & Industry  Communities  of  Practice 


Scarborough  (2006) 
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Figure  A.  1.2:  Reflective  Practice:  The  Scholarship  of  Teaching  - The  CEET  Model 
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The  Scholarship  of  Teaching  - Executive  Summary 

CEET  INITIATIVE  ON  TEACHING  AND  LEARNING  (CITE) 

Pilot  Results 
College  Portfolio 
May,  2007 

Jule  Dee  Scarborough,  Ph.D. 

The  Results  Summary  presents  the  overall  accomplishments  of  the  Initiative,  as  well  as  any 
important  notes  or  changes,  and  describes  all  educational  products  produced.  It  is  organized  into  the 
following  four  sections.  Each  section  includes  a brief  narrative  summary  followed  by  a chart  that 
itemizes  each  objective,  goal,  or  outcome;  these  are  followed  by  descriptions,  progress  to  date, 
quality  comments,  challenges  or  issues,  and  future  directions  or  changes. 

1.  Vision,  Mission  and  Operational  Objectives 

2.  Faculty  Development  Program  and  Faculty  Research 

3.  Faculty  Development  Program  Outcomes: 

4.  CITE  Faculty  Participant  Teaching  Portfolio  Summary 

1.  Vision,  Mission  and  Operational  Objectives 

Visions,  and  vision  statements,  are  not  short  term  in  nature.  But  any  initiative  can  generally  report 
progress  toward  attaining  a vision.  However,  since  visions  are  always  a “future  state”  and  are 
“living”  and  “dynamic,”  they  should  be  continuously  modified  to  reflect  higher  goals  of  excellence, 
each  time  raising  the  standards  to  achieve  something  more.  The  CITE  vision  to  become  a regional 
and  national  leader  for  the  Scholarship  of  Teaching  is  a major  and  complex  vision  to  achieve.  The 
immediate  mission  to  begin  to  move  towards  achieving  the  vision  is  one  of  creating  a faculty 
learning  community  (EC)  that  is  prepared  to  engage  formally  in  the  Scholarship  of  Teaching. 
Therefore,  we  began  with  a pilot  initiative  that  formally  involved  a small  group  of  faculty  members 
(7)  in  a very  intense  and  extended  faculty  development  program  to  prepare  them  for  the  Scholarship 
of  Teaching  - formal  experimental  classroom  research.  In  reviewing  the  chart  that  follows,  readers 
will  see  that  we  have  made  the  expected  progress  toward  each  objective,  exceeded  many  objectives 
and  goals,  and  now  understand  what  will  be  required  to  implement  the  faculty  development  program 
and  the  Scholarship  of  Teaching  more  broadly  across  the  college  and  its  four  departments.  We  are 
exactly  where  we  hoped  to  be  at  the  current  time:  an  interdisciplinary  and  functional  engineering 
and  technology  faculty  learning  community.  The  EC  has  successfully  completed  all  aspects  of  the 
program  and  engaged  in  one  semester  of  formal  experimental  classroom  research  on  teaching, 
student  assessment,  and  learning,  for  which  each  member  has  prepared  a research  manuscript  to 
submit  for  publication.  The  Dean  and  Faculty  Eearning  Community  are  planning  how  to  sustain 
their  accomplishments  (what  changes  to  make),  how  to  extend  the  EC  and  its  learning  and  research, 
and  how  to  disseminate  information  regionally  and  nationally  (including  writing  proposals  for 
additional  funding).  The  chart  below  itemizes  and  describes  the  status  of  each  aspect  of  vision, 
mission,  and  operational  objectives  and  goals,  pages  2-4.  (See  Portfolio  Introduction,  Section  A. I) 
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Table  A.2.1:  Mission,  Vision,  and  Operation  Objectives 


Legend: 

! Achieved/Excellent 

J Achieved  well,  or  + Moving  appropriately  towards 
P Partially  Achieved 
N Not  Achieved 


Type 

Description 

Progress  towards 
ultimate  Outcome 
and  Timeline 

Products 

Notes 

Future 
Direction 
& Changes 

Vision 

To  build  a regional  and  national 
reputation  for  the  Scholarship  of 
Teaching.  Begin  by  motivating 
an  interdisciplinary  learning 
community  of  faculty  to  own  and 
interpret  the  vision. 

Faculty  learning 
community  of  7 
professors  created  and 
functioning  well. 

+ 

T 

• 

N 

See 

College 

Portfolio 

Sections 

1 .Phase  II  of 
development 
for  original 
LC 

2.  Create 
second  CELT 
LC 

Mission 

To  build  (expand  and  sustain)  an 
interdisciplinary  team(s)  of  faculty 
who  understand  the  four  types  of 
scholarship  as  defined  by  Boyer 
(1990);  have  the  capability  of 
engaging  in  either  or  several  of  the 
types  of  scholarship;  are  stimulated 
to  engage  in  scholarship  activities, 
especially  research  on 
teaching/student  learning,  in  their 
disciplinary/  interdisciplinary 
classrooms. 

Pilot  completed. 

One  faculty  LC  in 
place  with  all  goals 
achieved  for  all  but 
one  faculty  member. 

r 

• 

See 

College 

Portfolio 

Sections 

Need  to  deepen  and 
sustain  the  LC  so  that 
the  faculty  can  continue 
to  a phase  II  of  learning 
and  deeper  commitment 
to  LC  and  the  four 
scholarships. 

1 .Phase  II  of 
development 
for  original 
LC 

2.  Create 
other  CEET 
LCs 

Operational  Objectives 

1.  Faculty 
Development 

1.  To  prepare  faculty  to  engage  in 
the  Scholarship  of  Teaching  and  its 
inherent  research  agenda. 

2.  To  prepare  a faculty  team  to  lead 
the  Scholarship  of  Teaching  in  the 
college. 

3.  To  engage  faculty  in  a program  of 
professional  development  on 
teaching  and  learning  and 

educational  research. 

Completed  well 

Completed  program  to 
date 

Completed  well 

t 

• 

• 

J 

See 

College 

Portfolio 

See 

Program 

Descrip- 

tion 

1.  Accomplished  well;  they 
wish  for  more  development 
to  extend  and  deepen 
understanding/  capability 

2.  Several  are  committed  to 
leading  the  LC;  all  desire 
to  continue  in  LC. 

3. Six  of  seven  completed 
every  aspect  of  the 
program  and  experimental 
research;  one  completed 
almost  everything. 

1 .Plan  and 
pilot  Phase  II 
of 

development 
for  original 
LC 

2. Estahlish 
sustaining  LC 
leader(s) 

3.  Determine 
LC  activities 

2.  Research 

1. To  study  the  effectiveness  of  the 
professional  development 
program 

2.  To  engage  in  the  Scholarship 
of  Teaching  and  study  the  effects 
of  selected  teaching  and  learning 
strategies  on  student  learning. 

3.  To  study  the  effects  of 
improved  traditional  testing  in 
conjunction  with  performance 
assessment. 

Formal  evaluation 
provided  strong 
evidence  of  success. 

All  professors 
executed  formal 
experimental 
classroom  research. 

Qualitative  evidence 
and  professors 
response  to 
performance 
assessment  was 
extremely  positive. 

T 

• 

f 

• 

y 

See 

College 

Portfolio 

See 

Individual 

Professor 

Research 

See 

Individual 

Professor 

Research 

1.  Professors  made  serious 
and  significant  changes  in 
courses,  teaching, 
assessment,  and  more. 

2.  Professors  learned  more 
about  the  difference 
between  experimental 
research  in  engineering  and 
technology  and  that  of 
education  as  social  science 
research. 

3.  Professors  responded 
positively  regarding 
professional  development 
on  test  analysis  and  the 
development  and  use  of 
performance  assessment, 
tasks/  rubrics. 

1 .Plan  and 
pilot  Phase  II 
of  develop 
ment  for 
original  LC 

2. Estahlish 
sustaining  LC 
leader(s) 

3.  Determine 
LC  activities 

4.  Initiate 
second 
faculty  LC 

5.  Offer 
program  for 
other  LCs 
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Legend: 

! Achieved/Excellent 

J Achieved  well,  or  + Moving  appropriately  towards 
P Partially  Achieved 
N Not  Achieved 


Type 

Description 

Progress  towards 
ultimate  Outcome 
and  Timeline 

Products 

Notes 

Future 
Direction 
& Changes 

3.  Quality 
Improvement 

1.  To  strengthen  teaching  and 
student  learning  across  CEET. 

2.  To  extend  research  beyond 
traditionally  accepted  definitions 
of  research  for  engineering  and 
technology  professors  in  the 
college. 

3.  To  build  confidence  in  the 
value  and  role  of  research  on 
teaching  and  learning  -the 
Scholarship  of  Teaching  and 
include  it  equally  alongside  the 
other  types  of  research  across 
the  college. 

Pilot  was  successfni 

End  of  conrse  evalnations 
were  approximately  the 
same,  but  it  is  too  soon  to 
expect  significant  change 
on  those;  they  were  not 
a variable. 

Student  responses  on  end 
of  course  questionnaire 
showed  significant 
change  in  responses  at 
.05  positive 

Faculty  executed  formal, 
experimental  classroom 
research  on 
teaching/learning 

Faculty  LC  is  interested 
in  continuing  the 
Scholarship  of  Teaching 
and  the  interdisciplinary 
LC 

Faculty  would  like 
additional  professional 
development 

+ 

t 

• 

+ 

r 

• 

+ 

See 

Course 

Analysis 

Products 

and 

Results 

See 

Research 

Models 

This  goal  is  one  that 
will  take  time  to 
achieve;  it  is  clear  that 
the  7 professors 
involved  in  this 
initiative  had  introduced 
and  implemented  use  of 
new  teaching  and 
learning  strategies, 
teaching  models,  styles; 
they  have  stimulated 
additional  student 
learning  styles;  new 
tests  and  performance 
assessments  were 
administered  for  student 
assessment;  also  they 
have  fully  executed 
experimental  classroom 
research  on  teaching 
and  learning.  They 
consider  research  on 
teaching  and  learning 
equal  to  that  of 
discipline  specific 
research. 

Revise 
program 
for  faculty 
preparation 

Expand 
classroom 
research  by 
existing  LC 
and  future 
LCs 

Further 
support  the 
Scholarship 
of  Teaching 
by  Dean  and 
Department 
Chairs 

Reward  for 
promotion 
and  tenure 

4.  Evaluation 

l.To  evaluate  the 

a.  faculty  development  model, 
program  content,  and  processes 

b.  classroom  research  results 

c.  outcomes  educational  products 

Feedback  and 
evaluation  of  program 
excellent 

Experimental 
classroom  fully 
executed 

New  educational 
products  excellent 

f 

• 

T 

• 

f 

• 

See  Final 
Reports 

All  aspects  of  the 
faculty  development 
model  and  program 
were  evaluated; 
classroom  research  was 
evaluated;  and  all 
educational  products 
were  evaluated  by 
program  leaders. 

Review  and 

revise 

program; 

Extend 

research; 

Continue 

evaluation 

5. 

Dissemination 

1.  To  disseminate  nationally: 

a.  faculty  development  model, 
program  content,  and  processes 

b.  classroom  research  results 

c.  evaluation  of  faculty 
development  program 

d.  faculty  improvement  results 

2.  products  through 

a.  ASEE  & NAIT  conferences 
and  journals 

b.  EE,  IE,  ME,  IT  journals 

c.  portfolio  (monograph) 

Complete  program, 
research,  results,  and 
evaluation  will  be 
available  on  CD  with 
Explanation: 

Submitted  to  ERIC 

Sent  to  all  engineer- 
ing/technology colleges 

Eaculty  submitted 
manuscripts  to 
journals  for 
publication  on 
individual  classroom 
research. 

Manuscript  submitted 
by  Dean  on  initiative 

T 

• 

t 

• 

t 

• 

• 

+ 

See 

Products 

in 

College 

Portfolio 

To  be  completed 
during  Summer  2007. 

Will  design 
national 
workshops 
and  seek 
funding  for 
participants 
from  NSF 
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Legend:  ! Achieved/Excellent 
J Achieved  well,  or  + Moving  appropriately  towards 
P Partially  Achieved 
N Not  Achieved 


Type 

Description 

Progress  towards 
ultimate  Outcome 
and  Timeline 

Products 

Notes 

Future 
Direction 
& Changes 

6.  Promotion 
& Tenure 
related  to 
Research: 

Scholarship  of 
Discovery 

Scholarship  of 
Integration 

Scholarship  of 
Application 

Scholarship  of 
Teaching 

To  redefine  scholarship  across  its 
departments  and  make  acceptable 
for  promotion  and  tenure  purposes, 
either,  ora  mix,  of  research 
productivity  across  the  types  of 
scholarship  as  defined  by  Boyer 
(1990). 

1.  To  submit  bylaw  content 
changes  to  include  the  Scholarship 
of  Teaching 

2.  To  review  and  seek  faculty 
approval  on  changes 

3.  To  implement  new  bylaws 

**We  determined  that  seeking  a 
change  in  the  bylaws  was  not 
necessary  to  confirm  acceptance  of 
the  Scholarship  of  Teaching  toward 
promotion  and  tenure,  merit,  and 
academic  reviews  leading  towards 
rewards.  The  College  confirmed  it 
as  acceptable. 

College  Personnel 
Committee 
confirmed  equal 
status  of 
Scholarship  of 
Teaching  alongside 
other  3 types  of 
research  towards 
promotion  and 
tenure. 

Departmental 
Chairs  confirmed 
equal  status  of 
Scholarship  of 
Teaching  as  above. 

Therefore,  no  need 
to  seek  bylaw 
changes. 

+ 

J 

Memos  from 
Dean, 

Department 

Chairs,  and 

College 

Council 

(Personnel 

Committee) 

Summer 

2007 

Educational 
Research  with  K- 
12  teachers  has 
historically  heen 
valued  for  tenure 
and  promotion. 

The  bylaws  do  not 
restrict  professors 
on  the  choice  of 
research;  all 
research  types, 
including  that  on 
teaching  and 
learning,  have 
heen  confirmed  as 
valued  towards 
tenure  and 
promotion.  We 
found  we  did  not 
have  to  rewrite  the 
bylaws. 

None 

needed 

Faculty  Goal 

To  engage  in  Scholarship  of 
Teachins.  either  as.  or  alongside 
their  other,  scholarship  interest(s). 

Accomplished;  See 
Research  Results 

f 

• 

Manuscripts; 
Research  Data 

Continue  by 

expanding 

research 

Scholarship 
of  Teaching 
Goal 

*To  adopt  standards  for  quality 
performance  in  scholarship: 

(Finished)To  include  the 
Scholarship  of  Teaching  as  an 
equally  acceptable  type  of 
scholarship  alongside  the  other 
types  of  scholarship,  discovery, 
integration,  and  application. 
(Boyer,  1990;  Glassick  et  al, 
1997). 

Research  Quality 
Standards  not  yet 
achieved. 

Equal  status  for 
Scholarship  of 
Teaching 
confirmed. 

N 

T 

• 

*This  is  planned 
for  the  next  phase. 

Not  yet  introduced; 

Will  occur  now 
that  SoT  is 
confirmed  as 
acceptable. 

Primary 
Objective 
for  second 
phase: 
2007-2008 

Faculty 

Development 

Goal 

To  design,  develop,  pilot, 
institutionalize,  and  sustain  a 
program  of  faculty  development  on 
teaching,  student  learning,  and 
educational  research  to  prepare 
faculty  to  engage  in  scholarship  of 
teaching  through  experimental  or 
action  research  in  the  classrooms. 

Faculty 
Development 
piloted;  Program 
evaluated  as 
excellent 

*Planning 
Sustainability  & 
Institutionalization 

T 

• 

+ 

* 

N 

Program 

Materials, 

Presentations, 

Worksheets, 

References, 

Handouts, 

Inventories, 

Feedback  & 

Evaluation 

Forms,  etc. 

Revise 

program 

and 

continue  to 
offer  for 
new  LCs 

Student  Goal 

To  develop  learners  in  the  truest 
sense,  meaning  that  students  leave 
us  with  such  excitement  about  what 
they  have  learned  that  they 
continue  to  seek  to  learn. 

Analysis  and  redevel- 
opment of  courses 
improved  content, 
assessment,  teaching; 
students  responses  on 
questionnaires  positive 

+ 

N 

Course 
changes  (e.g., 
syllabus, 
models,  etc.) 

Long  term  goal 
that  began  with 
this  initiative. 

Incorporate 

course 

assessments 

into 

research 
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2.  Evaluation  of  Faculty  Development  Program  and  Faculty  Research 

As  described  throughout  the  Introduction,  Program  Description,  Research,  and  Results  sections  of 
the  Portfolio,  two  prongs  of  research  and  evaluation  are  clear.  The  first  prong  of  research  and 
evaluation  focuses  on  the  Faculty  Development  Program  that  served  as  the  vehicle  to  prepare  faculty 
members  for  the  Scholarship  of  Teaching.  However,  because  our  ultimate  goal  was  to  design  and 
develop  an  integrated  program  for  faculty  development,  where  theory,  models,  strategies,  processes, 
techniques,  and  procedures  for  teaching  to  increase  student  learning  were  woven  together  into  a 
program  rather  than  separated  workshops,  we  decided  the  program  could  be  research  in  itself,  as  the 
model,  processes,  and  content  were  implemented  differently  than  usual  for  a university  level  faculty 
development  process.  Therefore,  we  structured  the  study  of  the  program  as  formal  research  and 
evaluated  it  formally. 

Another  critical  aspect  of  our  strategy  was  that  the  classroom  implementation  of  new  teaching  and 
learning  strategies  and  the  experimental  research  both  be  an  integral  component  of  the  faculty 
development  program;  therefore,  the  program  took  professors  from  building  a foundation  on 
teaching  and  learning  through  product  development,  new  instructional  choices  about  models  and 
strategies,  and  into  the  classroom  for  implementation.  Educational  research  was  also  a program 
component  of  the  faculty  development  program  and  prepared  the  faculty  for  an  initial  attempt  at 
experimental  research  in  the  classroom.  It  was  important  to  include  the  actual  engagement  of 
classroom  research  as  one  of  the  program  components  to  also  follow  into  the  classroom.  Therefore, 
research  was  designed,  methodology  and  procedures  were  determined,  and  the  professors  engaged 
in  their  first  attempt  at  formal  experimental  classroom  research  during  the  implementation  and 
“research  semester”  as  a program  component. 

We  learned  many  lessons  as  a result  of  our  history  of  providing  professional  development  for  math, 
science,  technology  and  English  teachers,  so  we  applied  those  lessons  to  higher  education. 

(See  strategicalliance.niu.edu.)  For  example: 

1.  Separated  workshops  without  support  to  follow  through  with  classroom  implementation 
usually  result  in  fewer  actual  changes  in  instruction  or  teaching  and  student  learning  in  the 
classroom. 

2.  Our  programs  with  teachers  were  usually  18-21  full  days,  followed  by  supported  classroom 
implementation.  Without  what  some  would  call  extended  time,  very  little  understanding  was 
realized. 

3.  Even  following  development  with  classroom  implementation  does  not  go  far  enough  to 
ensure  sustainability  without  “closing  the  loop”  and  engaging  together  to  analyze  what 
models,  processes,  strategies,  new  products,  new  practices  were  ACTUAFFY  implemented 
or  used,  and  then  discussing  what  worked  and  what  did  not,  identifying  changes  for  the  next 
time,  and  formalizing  those  changes  so  instructional  practices  and  decisions,  or  behaviors, 
continue  to  change  and  evolve. 

4.  Support  is  critical.  Formal  leadership  is  critical,  as  is  follow  up  by  leaders  to  ensure  that 
results  are  documented,  lessons  learned  are  recorded,  and  dissemination  occurs,  beginning 
with  EC  members  and  then  across  FCs  internally,  regionally  and  nationally. 
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Faculty  development  is  a complex  and  dynamic  process.  Program  content  is  not  easily  determined, 
and  good  needs  assessment  and  definition  of  the  “current  reality”  are  essential  before  determining  a 
vision,  the  initial  mission,  and  then  the  objectives  or  outcomes. 

The  chart  below  itemizes  and  describes  each  variable  for  the  faculty  development  program  research. 
The  faculty  members’  classroom  research  was  one  variable  to  determine  the  success  of  the  overall 
faculty  development  program.  Each  variable  is  followed  by  its  status  to  date,  notes,  and  future 
changes.  Consult  the  Research  Section  for  more  detail  on  the  research  methodology,  procedures,  and 
literature  on  faculty  development  programs.  That  section  also  more  thoroughly  discusses  the 
individual  faculty  members’  research  designs  and  results.  (See  Portfolio  Sections  A. 3 and  B.0-12) 
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Table  A.2.2:  Faculty  Development  Program  and  Faculty  Research  Results  Summary 


Type 

Description 

Status 

Notes 

Future 

Changes 

Student 
End  of 
Semester 
Questionnaire 

Extensive  questionnaire  on 
all  aspects  of  teaching  and 
learning  within  student 
course  context. 
Administered  for  2005 
course  followed  by 
administration  in  2006 
experimental  course; 

Was  used  for 

comprehensive  perspective. 

2005 

Significant 
difference  at 
.05  level 

2006 

Significant 
difference  at 
.05  level 

It  was  developed  to  be  comprehensive; 
therefore,  it  was  extremely  long.  It  was 
developed  to  be  used  as  a bank  of  questions 
from  which  items  could  be  chosen  for  more 
specific  purposes.  We  choose  to  administer 
it,  knowing  that  it  was  too  long. 

Therefore,  some  would  question  the  results 
for  the  2006  administration.  However, 
professors  have  faith  that  students  took  it 
seriously.  This  can  be  resolved  by  using  it 
as  a bank  of  questions  from  which  to 
choose. 

Use  as  a bank 
of  items  from 
which  to 
choose  for 
specific 
research  focus 
and  purpose. 

Faculty 

(Student) 

End  of 

Semester 

Questionnaire 

Same  questionnaire;  we 
used  it  to  see  how  the 
professors  would  respond, 
answering  from  the 
perspective  of  their  students, 
about  themselves.  Timing 
same  in  2005  and  2006  as 
students.  Was  used  for 
comprehensive  perspective. 

2005 

Significant 
difference  at 
.05  level 

2006 

Significant 
difference  at 
.05  level 

Professors  indicated  that  they  took  it 
seriously  for  both  administrations. 

Administer  for 
same  purpose 
and  in  same 
manner; 

Or  they  could 
complete  the 
same 

abbreviated 
version, 
described 
above,  as 
students, 
focusing  only 
on  particular 
research  or 
items  of 
interest. 

Faculty  Self 
Competency 
Questionnaire 

Abbreviated  and  focused 
version  of  the  full 
questionnaire  above.  Focus, 
however,  was  different. 
Professor  viewpoint  was  to 
assess  his/her  own  level  of 
competency  regarding 
knowledge  and  skills  on 
teaching  and  learning. 

2005 

Significant 
difference  at 
.05  level. 

2006 

Significant 
difference  at 
.05  level 

Worked  well. 

No  changes 
suggested. 

Could  be 
modified  as 
needed  for 
particular  foci. 

Program 

Content 

Knowledge 

Assessments 

Pre-  and  Post-assessment  of 
knowledge  in  each  program 
component  area. 

Significant 
difference  at 
.05  level 

Worked  well.  Format  and  items  seem  to 
result  in  adequate  and  appropriate  feedback 
to  use  to  monitor  program  quality  and 
faculty  response  to  program  and  also  to 
determine  when  to  “tweak”  program. 

No  changes 
suggested. 

Program 
Feedback  & 
Evaluation 

Program  Component 
feedback  that  collectively 
served  as  one  aspect  of 
program  evaluation 

Excellent 

Professors  were  very  pleased  with  the 
program  and  its  results  overall.  They 
suggested  shortening  the  program  to  fewer 
days,  while  also  requesting  more  time  on 

1 . Educational  research 

2.  Add  the  following  to  program: 

a.  formal  student  teaming 

b.  student  conflict  & classroom  mgmt. 

Rework 
program  to 
reduce  1 8 days 
to  fewer  days 
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Type 

Description 

Status 

Notes 

Future 

Changes 

Student 

Assessment 

Test 

Analysis 

Professors  analyzed  the 

2005  midterm  and  final 
exams;  identified 
strengths/weaknesses;  used 
to  develop  new  exams; 

Professors  analyzed  the  new 

2006  midterm  and  final 
exams  used  during 
experimental  research 
semester 

Analyses 
performed 
and  used 
diagnostic- 
cally 

Worked  well.  Use  of  analyses  as  diagnostic 
tools  need  to  be  strengthened;  faculty 
members  need  to  implement  test  analyses 
routinely,  or  at  least  periodically,  to  analyze 
quality  of  tests  and  instruction  with  goal  of 
increasing  student  learning  and  ability  to 
provide  evidence  of  learning. 

No  changes 
suggested 

Performance 

Assessments 

No  baseline  analysis,  as 
professors  did  not  have 
performance  tasks/rubrics. 

Each  professor  developed  3 
complex  performance  tasks 
with  corresponding  rubrics 
and  used  them  during  the 
experimental  research 
semester 

Excellent 

Worked  well.  Professors  did  review  the 
results  of  using  formal  performance  tasks 
and  rubrics  for  the  first  time.  They  need  to 
continue  such  a review,  similar  to  test 
analysis  above,  for  diagnostic  purposes  to 
determine  what  worked  well  and  what 
needs  to  improve  or  change  in  the 
performance  tasks,  rubrics  or  instruction,  to 
increase  student  learning  and  ability  to 
provide  evidence  of  learning. 

No  changes 
suggested 

Professors’ 

Teaching 

Portfolios 

Each  professor  developed 
his/her  teaching  portfolio, 
which  provided  evidence  of 
his/her  learning  and 
performance.  The  portfolio 
was  used  as  assessment  as 
learning 

Excellent 

Worked  well;  each  professor  has  full 
documentation  of  what  he/she  learned, 
developed,  and  evidence  of  change  and  growth; 
the  portfolio  also  provides  a record  of  what 
he/she  has  yet  to  try,  so  it  is  a dynamic,  living 
record  where  they  continue  to  keep  track  of  their 
evolution  of  the  changes  - e.g.  teaching,  instruc- 
tional practices,  course  content,  research  goals 

No  changes 
except  to  keep 
them  going  as 
ongoing  records 
of  learning, 
growth,  and 
change;  to  set 
new  goals 

Professors’ 

Research 

Each  professor  engaged  in 
the  Scholarship  of  Teaching 
during  an  experimental 
research  semester. 
Collectively,  they 
experimented  with  new 
course  structures;  new 
syllabi;  performance 
assessment;  new  teaching 
models.  Individually,  they 
researched  particular  aspects 
of  teaching/learning  specific 
to  their  individual  interests. 

Excellent  as 
their  first 
formal 
engagement 
in 

educational 
research  in 
the 

classroom. 

Worked  well.  Needs  to  be  followed  by 
further  professional  development  on 
educational  research  and  teaching  and 
learning  to  deepen  understanding  and 
knowledge  and  skills  on  the  Scholarship  of 
Teaching. 

We  did  not  build  the  variable  - student 
knowledge  gain  on  course  content  as  a 
whole  - into  the  research  design.  We  felt  we 
should  have.  Therefore,  we  suggested  that 
faculty  members  develop  a pre  and  post  test 
for  the  course  as  a whole  and  administer  it 
to  determine  knowledge  gain  for  the  course. 

No  changes 
suggested  for 
first  phase. 

Add  a second 
phase  to 
deepen 

knowledge  and 
understanding. 

Add  pre  and 
post  test  for 
course. 

Professors’ 

Manuscripts 

Evidence  of  each 
professor’ s Scholarship  of 
Teaching  in  a research 
manuscript  format;  to  be 
submitted  to  journals. 

Eine  for 
confirming 
formal 
classroom 
research 

Manuscript  quality  ranged  from  excellent  to 
approproprite  drafts;  2/7  need  more 
consideration,  but  documented  the  research  fine. 
All  professors  need  to  close  the  loop  by  identify 
future  changes  and  research  questions.  Some 
can  be  submitted  to  journals  in  current  state; 
others  need  revision.  But,  all  provided  high 
quality  evidence  of  the  classroom  research. 

1 . Close  the 
Loop 

2.  Add  literature 
to  establish 
studen  basis 

3.  Add 
literature  to 
consider  results 

Standard 
Student 
Evaluations 
and  Course 
Grades 

Initially,  we  thought  that  we 
could  use  “change  in  student 
evaluations”  and  “course 
grades”  as  part  of  program 
research  & evaluation. 
However,  we  did  not  for-mally 
include  as  variables. 

No  Status 

Eeel  that  these  two  variables  should  be 
incorporated  in  next  round  of  program 
implementation  and  used  as  variables  in 
research  and  for  evaluation  purposes. 

Add  and 
formalize 
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3.  Faculty  Development  Program  Outcomes: 

4.  CITL  Faculty  Participant  Teaching  Portfolio  Summary 


Although  the  chart  below  identifies  all  of  the  faculty  development  program  outcomes  that  we 
desired  to  achieve,  there  are  few  notes  or  change  comments.  The  program  worked  well;  all  outcomes 
were  either  achieved  or  exceeded.  As  a direct  result  of  faculty  members’  dedication,  commitment, 
and  hard  work,  and  their  tolerance  for  the  “ever  expanding  requests”  to  try  something  new  as  ideas 
evolved  throughout  the  program,  their  list  of  accomplished  outcomes  is  phenomenal.  Also  the 
quality  of  their  work  was  outstanding,  especially  considering  the  number  of  products,  complexity  of 
products,  and  level  of  detail  and  tediousness  of  some  of  the  analyses,  which  were  then  summarized 
into  other  culminating  analyses.  The  analysis  process,  although  appropriately  scaffolding,  could 
have  seemed  unnecessary  and  very  much  like  a “Never  Ending  Story”;  however,  they  did  it  ALL! ! 

The  Outcomes  chart  immediately  below  presents  the  list  of  outcomes;  these  are  described  more 
completely,  and  with  worksheets  in  the  Program  Description  Section  of  the  Portfolio. 

The  Outcomes  Chart  is  followed  by  the  CEET  Teaching  Portfolio  Chart,  which  provides  a list  of 
each  type  of  product  or  educational  decision  made  by  each  professor.  This  is  a culminating  college 
chart  reflecting  the  contents  of  the  teaching  portfolio  for  each  professor  who  participated.  The  two 
charts  reveal  the  breadth  and  depth  of  the  program,  all  products,  and/or  instructional  or  behavioral 
decisions.  There  are  minor  changes,  additions,  or  suggestions  for  the  future,  but  overall  everything 
was  accomplished  to,  or  exceeded,  expectations.  See  Portfolio  Sections  A.5  and  B.0-B.12. 
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Table  A. 2. 3:  Faculty  Development  Program  Outcomes 


Legend: 

! Achieved/Excellent 

J Achieved  well,  or  + Moving  appropriately  towards 
P Partially  Achieved 
N Not  Achieved 


Outcomes 

Status 

Notes 

Future  Changes 

I.  To  analyze  each  existing  course  to 

a.  determine  appropriate  content  knowledge 
for  achieving  ABET/TAC/NAIT  standards 
or  student  learning  outcomes 

b.  determine  knowledge  content  priority: 
major,  secondary,  other  or  minor 

c.  determine  how  knowledge  fits  into 
Bloom’s  Taxonomy  Knowledge  Dimension 

d.  determine  the  embedded  general 
education  goals 

e.  determine  appropriate  teaching  models 
and  styles 

f.  determine  which  student  learning  styles 
are  being  engaged 

g.  determine  the  levels  of  Bloom’s 
Taxonomy  Dimension  of  Learning  being 
achieved 

h.  determine  the  levels  of  Dale’s  Cone  of 
Learning  being  achieved 

i.  determine  strengths  and  weaknesses  of  the 
course 

j.  determine  strengths  and  weaknesses  of 
instruction 

k.  determine  strengths  and  weaknesses  of 
syllabus 

Completed;  Excellent 

T 

• 

T 

• 

T 

• 

f 

• 

f 

• 

f 

• 

T 

• 

f 

• 

f 

• 

f 

• 

f 

• 

See  Results  in 

Analysis 

Reports 

Reduce  program 
component  time 

II.  To  analyze  all  tests  to 

a.  determine  the  overall  quality  of  the  test 

b.  determine  the  overall  quality  of  test  items 

c.  identify  strengths  and  weaknesses  of 
existing  tests 

d.  map  test  relationship  to  course  outcomes 

e.  map  test  items  to  course  outcomes 

f.  analyze  other  assessments  for  quality 

Completed;  Excellent 

f 

• 

f 

• 

f 

• 

f 

• 

T 

• 

T 

• 

See  Results  in 

Analysis 

Reports 

No  changes 
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Legend: 

! Achieved/Exeellent 

7 Achieved  well,  or  + Moving  appropriately  towards 
P Partially  Achieved 
N Not  Achieved 


Outcomes 

Status 

Notes 

Future  Changes 

III.  To  redevelop  course  outcomes  that 
directly  link  to  ABET/  TAG/  NAIT 

a.  redevelop  the  course  outcomes  and  map 
relationship  to  ABET/TAC/NAIT 

b.  break  down  outcomes  in  outline  form: 
major,  secondary,  minor  levels 

c.  identify  knowledge  according  to  Bloom’s 
Knowledge  Dimensions 

d.  identify  embedded  general  education 
goals 

e.  map  outcomes  to  Bloom’s  Dimension  of 
Learning  levels 

f.  map  outcomes  to  Dale’s  Cone  of  Learning 
levels 

Completed;  exeellent 

T 

• 

T 

• 

f 

• 

T 

• 

f 

• 

f 

• 

See  Results  in 

Analysis 

Reports 

See  Course 
Syllabi  Report 
reflecting 
content, 
schedule, 
outeomes,  etc. 

No  changes 

IV.  To  redevelop  tests  that  directly  link  to 
course  outcomes  and  ABET/  TAC/NAIT 

a.  create  a table  of  speeifications 

b.  develop  a bank  of  diverse  test 
items,  multiple  items  for  each 
outcome 

1 . multiple  choice 

2.  true/false 

3.  short  answer 

4.  matching 

5.  problems 

e.  assemble  two  comprehensive  tests 

1 . midterm  examination 

2.  final  examination 

Completed;  exeellent 

f 

• 

f 

• 

T 

• 

T 

• 

See  Portfolio 
and  Results 
Report 

No  changes 

11 


Legend: 

! Achieved/Exeellent 

7 Achieved  well,  or  + Moving  appropriately  towards 
P Partially  Achieved 
N Not  Achieved 


Outcomes 

Status 

Notes 

Future  Changes 

V.  To  design  and  develop  a more 
diverse  and  balanced  student 
assessment  plan 

a.  develop  3 complex  performance 
tasks  with  corresponding  rubrics 

1 .task  and  rubric  that  correspond 
with  the  midterm  exam 

2. task  and  rubric  that  correspond 
with  the  final  exam 

3.  task  and  rubric  to  further  enhance 
the  more  balanced  assessment  plan 

d.incorporate  student  self-assessment 
using  rubrics 

b.  develop  other  types  of  student 
assessments  to  further  diversify  and 
balance  the  course  assessment  plan; 
choose  from  or  determine: 

1 . quizzes 

2.  projects 

3.  case  studies 

4.  papers 

5.  reports 

6.  literature  reviews 

7.  design  problems 

8.  presentations 

9.  concept  mapping 

10.  team  projects 

1 1 . field  experiences 

12.  simulations 

13.  portfolios 

c.  employ  student  self-assessment 
procedures  on  particular  or  all 
assessments 

Completed; 

excellent 

f 

• 

T 

• 

f 

• 

f 

• 

f 

• 

T 

• 

P 

N 

See  Assessment 
Plans  in  Results 
Reports 

Assessment  plans 
became  more 
multifaceted  and 
balanced  by 
redeveloping  better 
traditional  tests,  better 
problems,  but  most 
importantly  by  adding 
the  performance 
assessments  and  rubrics. 
Some  also  added  other 
assignments  as 
assessment  more 
formally.  Some  did  not. 

At  this  stage, 
adding/redeveloping  the 
tests  and  adding  and 
developing  the  3 
performance 
assessments  and  rubrics 
were  the  goals. 

Each  professor  also 
mapped  the  course 
assessment  plan  and 
connected  each  test,  test 
item,  and  assessment 
and  rubric  criteria 
directly  to  student 
learning  outcomes. 

5fc 

Although  professors 
were  asked  to  use  a 
student  self-assessment 
process  with  the 
performance 
tasks/ruhrics,  only  a few 
actually  tried  it;  others 
did  not.  Student  self- 
assessment  needs  more 
focus,  thought,  and 
process  time  for 
professors  to  more  fully 
implement 

No  changes 

Next  phase, 
broaden  plan  to 
include  more 
assessments  with 
grading  criteria 
specified 

**Incorporate 
student  self- 
assessment  as  a 
process;  use  with 
performance; 
consider  adding 
portfolio 
assessment 
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Legend: 

! Achieved/Exeellent 

7 Achieved  well,  or  + Moving  appropriately  towards 
P Partially  Achieved 
N Not  Achieved 


Outcomes 

Status 

Notes 

Future  Changes 

VI.  To  reconsider  grades,  grading  criteria 
and  processes 

a.  no  curving  of  grades 

b.  determine  grading  criteria 

c.  determine  scoring  protocols 

d.  implement  rubrics 

e.  implement  student  self-assessment 

f.  determine  formal  course  assessment 
grading,  scoring  structure 

Completed; 

Improved 

J 

J 

J 

f 

• 

PN 

J 

See  Course 
Syllabi  in  Results 
Reports 

Although  professors 
were  asked  to  use  a 
student  self-assessment 
process  with  the 
performance 
tasks/rubrics,  only  a few 
actually  tried  it;  others 
did  not.  Student  self- 
assessment  needs  more 
focus,  thought,  and 
process  time  for 
professors  to  more  fully 
implement 

No  changes; 
extend  by 
professors  as  they 
learn  and  revise 
courses 

VII.  To  reconsider  other  instructional 
decisions  by  increasing  the  repertoire  of 
options: 

a.  choose  a broader  repertoire  of  teaching 
models  to  use  in  the  redeveloped  course 

b.  choose  a broader  repertoire  of  teaching 
styles  to  use  while  teaching  the  redeveloped 
course 

c.  provide  a wider  range  of  learning 
opportunities  that  engage  a more  diverse 
range  of  learning  styles 

d.  consider  multiculturalism  and  its  effect  on 
student  learning  and  planning  instruction 

e.  consider  student  motivational  factors  in 
making  instructional  decisions 

f.  consider  student  perception  factors  in 
making  instructional  decisions 

g.  consider  improvements  of  learning 
environment  and  learning  space 
arrangements  (second  phase  of  program) 

Completed; 

Improved 

J 

J 

J 

J 

J 

J 

N 

See  Course 
Analyses,  Syllabi, 
and  Results 
Reports 

*Some  professors 
had  to  change 
learning  space  to 
incorporate  new 
performances,  but 
this  was  not 
addressed 
formally  in 
program 

As  professors 
keep  modifying 
courses,  they  will 
continue  to  add  to 
each  type  of 
repertoire, 
teaching  models, 
styles,  broadened 
learning  styles 
resulting  from 
additional  models 
and  styles 

VIII.  Determine,  design,  develop... finalize 

a.  contextual  curricula 

b.  learning  activities 

c.  group  or  team  learning  and  assessment 
processes 

Completed; 

Improved 

J 

J 

J 

See  Course 
Analyses,  Syllabi, 
and  Results 
Reports;  also  see 
Professor 
Research  Reports 

Ongoing  learning, 
change,  and 
research 
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Legend: 

! Achieved/Exeellent 

7 Achieved  well,  or  + Moving  appropriately  towards 
P Partially  Achieved 
N Not  Achieved 


Outcomes 

Status 

Notes 

Future  Changes 

IX.  Redesign  and  develop  new  course 
syllabus  incorporating  the  following 
categories: 

a.  professor,  graduate  assistant  contact  information 

b.  catalog  course  description 

c.  course  purpose 

d.  course  requirements:  text,  datebook,  curricular 
course  packets,  etc. 

e.  course  pre-  or  co-requisites 

f.  expected  computer  use,  knowledge,  skills, 
software,  etc. 

g.  student  learning  outcomes,  identifying 
embedded  general  education  goals,  and  showing 
connection  to  ABET/TAC/NAIT  outcomes  with 
links  to  assessments 

h.  course  schedule/timeline  showing  course 
weeks/days,  topics,  activities,  due  dates,  lectures, 
tests,  projects,  fieldtrips,  etc. 

i.  course  requirements:  list  of  assessments  and 
point,  percentage,  structure,  etc. 

j.  grading  structure 

k.  academic  misconduct  or  cheating  policy 

l.  professor’s  role;  graduate  assistant’s  role 

m.  professor’s  notes:  particular  notes  about 
expected  behavior,  rules,  tardiness,  absenteeism, 
cell  phones,  late  assignments,  etc. 

n.  support  services  available  to  students,  (e.g.. 
Writing  Center,  tutorial  services,  accessibility 
services,  etc.) 

o.  course  references 

p.  course  requirements  explanation  - description 
of  each  type  of  assignment 

q.  course  requirements  check  off  - list  of  all 
assignments,  projects,  activities  with  point, 
percentage,  scoring,  or  grading  information  so 
students  can  keep  track  of  their  progress  in  course 
more  easily. 

Completed; 

greatly 

improved 

J 

J 

J 

J 

J 

J 

J 

J 

J 

J 

J 

J 

J 

J 

J 

J 

J 

J 

Each  professor 
needs  to  now 
modify,  clean  up, 
and  finalize  visual 
format  of  new 
syllabi 

Perhaps  provide 
graphic  template 
for  professors  so 
the  final  product 
would  reflect: 

Cleaned  up  new 
formats 

Added  missing 
sections  that  were 
recommended  by 
program  leaders 
and  literature  for 
inclusion 

Use  of  graphics  to 
organize;  boxing, 
etc. 
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Legend:  ! Achieved/Excellent 

7 Achieved  well,  or  + Moving  appropriately  towards 
P Partially  Achieved 
N Not  Achieved 


Outcomes 

Status 

Notes 

Future  Changes 

X.  Conduct  classroom  research  on 
teaching  and  learning 

a.  design  research 

b.  select  methodology 

c.  conduct  experiment 

d.  collect  data 

e.  analyze  and  interpret  data 

f.  develop  conclusions  and 
recommendations 

Completed; 

excellent 

J 

J 

J 

J 

J 

J 

Professors 
engaged  in  formal, 
experimental, 
classroom 
research.  They 
learned  about  the 
complexities  of 
doing 

experimental 
research  in  a 
social  context 
versus  an 
engineering  or 
technology 
context.  Social 
science, 
educational 
research,  is  more 
complex  in  nature. 

They  produced 
research 
manuscripts  for 
publication. 

Ongoing  learning, 
change,  research, 
and  publications 
IF  sustained 
initiative  in  the 
college. 

g.  prepare  manuscripts  for  publication 

Evidence  of  each  professor’s 
Scholarship  of  Teaching  in  a research 
manuscript  format;  to  be  submitted  to 
journals. 

J 

Fine  for 
confirming 
formal 
classroom 
research 

Manuscript  quality 
ranged  from 
excellent  to 
approproprite  drafts; 
2/7  need  more 
consideration,  but 
documented  the 
research  fine.  All 
professors  need  to 
close  the  loop  by 
identify  future 
changes  and 
research  questions. 
Some  can  be 
submitted  to 
journals  in  current 
state;  others  need 
revision.  But,  all 
provided  high 
quality  evidence  of 
the  classroom 
research. 

1 . Close  the  Foop 

2.  Add  literature 
to  establish  studen 
basis 

3.  Add  literature 
to  consider  results 
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Table  A.2.4:  Teaching  Portfolio  Assessment  Chart,  January  28,  2007-  CITL  Faculty  Development  Program 


Portfolio  Product  (Artifact)  Content 
(See  Sections  of  information  following  this  summary) 

RM 

RR 

*AA 

BT 

IM 

BC 

*AG 

Self  Assessment  Baseline: 

1 . Student  Questionnaires  (f05  & f06) 

1 .  Professor  completion(s)  of  Student  Questionnaire  (f05  &f06) 

2.  Professor  completions  of  Self  Competency  Questionnaire 

(Feb.06,  May06,  Dec.06) 

3.  Program  Components  Assessments  (8) 

4.  Standard  Departmental  Course  Evaluations  (f05  & f06) 

4.  Student  Grades  & End  of  Semester  Grades  (f05  & f06) 

J 

J 

J 

J 

J 

J 

J 

5.  Course  Analysis: 

5al._Course  Outline,  Embedded  Gen  Ed,  Content  Priorities 
5a2._Course  Content  Analysis  by  TM,TS,  LS,  Bl,  Dale,  etc. 

5b.  _ Instr.  Design  GAPS  Analysis  on-  TM,  TS,  LS,  B.  D 
5c.  _ Instructional  GAPS  Summary 
5d.  _ ABET/TAC/NAIT  SLO  by  Bloom’s  Analysis 
5e.  _ Course  Content  Schedule 

5f._  Teaching  Models+Cooperative  Learning+Study  Chart+TM  graphic 
5g._  Course  Calendar  by  TM,  TS,  LS,  B,  D 

J 

J 

J 

J 

J 

J 

J 

***Student  Learning  Styles  Inventory:***  (NOT  REQUIRED) 
Kolb  (Extra  professional  effort  on  part  of  professors) 

Felder  (Extra  professional  effort  on  part  of  professor) 

KJ 

KJ 

NA 

NA 

NA 

NA 

Multifaceted  Assessment  System: 

5h. Multifacted  Assessment  Plan  Graphic,  showing  course  assessments 

5i. Test  and  Test  Items  by  SLO  Chart 

5j.  Assessment  Analysis  by  Bloom  (Chart) 

J 

J 

J 

J 

J 

J 

J 

6.  Traditional  Objective  Tests:: 

Test  Analysis  ("Midterm  and  Final  Exam) 
Table  of  Specifications  tnot  included) 
Test  Item  Bank  tnot  included) 

7.  New  Midterm  Exam 

7.  New  Final  Exam 

8.  Diagnostic  Write  Ups  (MT  & F) 

J 

J 

J 

J 

J-  Partial 

Midterm  Analysis 

X Nd  analysis  far 
Final  Exam 

X Diagnostics 
No  Analysis  in  Report 

J 

J 

Performance  Assessment  & Rubrics: 

7.  3 Complex  Performance  Assessments  with  multiple  tasks 

embedded 

7.  3 Rubrics,  one  to  score  each  Performance  Assessment 

( And  to  be  used  with  students  to  establish  standards  up  front) 

8.  Diagnostic  Write  Ups  tPA  1.2.3) 

* Copies  of  Students  Rubrics  ("Hardcopies  on  file) 

7.  Electronic  copies  of  tests  and  PAs  & Rubrics 

other  Assessments  of  Individual  Choice:  Yes  for  All 

J 

J 

J 

J 

>/-Paptial 

Did  not  seem  to 
ose  PA  2,3 

X 

J 

J 

9.  Student  Centered  Course  Syllabus: 

All  new  components  and  check  off  list 

J 

J 

J 

J 

J 

J 

J 

10.  Professors’  Research: 

Completed  Data  Forms  (including  data  on  MT,  F,  PA1,2,3) 

Research  Results  Reports 

J 

J 

J 

J 

Partial  ^Midtarm 
X No  Final-  Robrics 

J 

J 

J 

12. Teaching  Portfolio  Assessment  Questionnaire 

13  Teaching  Models  Self  Assessment 

14  Teaching  Styles  Self  Assessment 

15  Student  Learning  Style  Opportunities  Assessment 

16  Outcomes  Achieved  as  Planned  by  Bloom  & Dale  Assessment 

J 

J 

J 

J 

J 

J 

J 

17.  Manuscript  to  be  submitted: 

Draft 

Final  Version  to  be  submitted  to  journal  (Mav.  2007) 

J 

J 

J 

J 

Partial 

© Final  Report, 
not  article 

J 

J 

Legend:  J = akay  X = still  needed  NA=  net  required  See  Results  in  Pertfelie  Section  B.I-12 
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Scholarship  of  Teaching 
What  Counts  as  Research? 

Jule  Dee  Scarborough,  Ph.D.  and  Jerry  Gilmer,  Ph.D. 

The  richness  of  faculty  talent  should  be  celebrated,  not  restricted.  Only  as  the 
distinctness  of  each  professor  is  affirmed  will  the  potential  of  scholarship  be  fully 
realized.  Surely,  American  higher  education  is  imaginative  and  creative  enough  to 
support  and  reward  not  only  those  scholars  uniquely  gifted  in  research  but  also  those 
who  excel  in  the  integration  and  application  of  knowledge,  as  well  as  those  especially 
adept  in  the  scholarship  of  teaching.  Such  a mosaic  of  talent,  if  acknowledged,  would 
bring  renewed  vitality  to  higher  learning  and  to  the  nation.  (Boyer,  1990,  p.  27) 

Educational  research  is  certainly  distinct  from  what  Wankat,  Felder,  Smith,  and  Oreovicz  call 
engineering  research  (as  cited  in  Huber  & Morreale,  2002).  Wankat  et  al.  point  out  that 
engineering  research  generally  attempts  to  understand  physical  mechanisms  and  phenomena 
or  to  identify  the  effects  of  specific  and  unambiguously  defined  variables  on  the  “behavior  of 
a process  or  system.”  Research  in  the  physical  world  involves  clear  variables  that  are 
measured  with  what  are  generally  accepted  to  be  unequivocally  reliable  and  valid  procedures. 
On  the  other  hand,  social  science  research,  of  which  educational  research  is  a subset, 
involves  less  well  defined  variables  and  less  reliable  and  valid  measurement  procedures. 
Social  science  research,  however,  has  a rich  history  of  development  and  growth  that  has 
produced  paradigms,  procedures,  and  methodologies  very  much  like  those  in  the  physical 
sciences,  which  when  understood  and  applied  appropriately  have  led  to  significant  and 
important  findings  and  advances  in  social  science  disciplines,  such  as  education. 

Wankat  et  al.  also  indicate  that  the  ultimate  goal  in  educational  research  is  to  improve 
learning,  but  “it  is  difficult  to  find  two  engineering  educators  who  would  agree  on  what  that 
means”  (as  cited  in  Huber  & Morreale,  2002).  Although  this  is  an  overstatement,  the  real 
issue  is  that  often,  due  to  ambiguities  in  the  meanings  of  broadly  labeled  variables  like 
“learning,”  social  science  researchers  must  define  the  variables  they  are  studying  in  the 
context  of  the  current  research  so  readers  can  consider  the  researchers’  definitions  when 
judging  the  extent  to  which  they  will  generalize  and  apply  the  results.  When  the  variables, 
and  their  measurement  procedures,  are  appropriately  and  operationally  defined  by  the 
researcher  and  when  accepted  research  designs  and  methodologies  are  employed,  the 
technical  quality  of  the  research  is  generally  not  in  question.  The  issue  for  consumers  of 
educational  research,  as  well  as  other  researchers,  is  essentially  the  degree  to  which  the 
primary  researchers’  operational  definitions  of  the  studied  variables  conform  to  the 
environments  and  contexts  of  the  consumers  and  other  researchers.  This  issue  is  not  under 
the  control  of  the  primary  researcher.  This  is  very  different  for  engineers  and  technologists. 

One  of  the  most  respected  sources  discussing  social  science  methodologies  is  Experimental 
and  Quasi-Experimental  Designs  for  Research  (Campbell  & Stanley,  1963).  Their  booklet 
“is  committed  to  the  experiment  as  the  only  means  for  settling  disputes  regarding  educational 
practice,  as  the  only  way  of  verifying  educational  improvements,  and  as  the  only  way  of 
establishing  a cumulative  tradition  in  which  improvements  can  be  introduced.”  Although 
formal  experiments  may  not  be  the  only  way  to  advance  and  improve  the  field  of  education. 
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it  is  clear  that  Campbell  and  Stanley  are  solidly  committed  to  the  experimental  paradigm  as 
an  effective  methodology  in  educational  research. 

Campbell  and  Stanley  (1963)  discuss  sixteen  different  experimental  designs,  only  three  of 
which  are  classified  as  true  experimental  designs.  The  three  true  experimental  designs 
control  for  all  eight  of  the  potential  sources  of  internal  invalidity.  The  internal  validity  of  an 
experiment  is  essentially  the  extent  to  which  the  results  of  the  experiment  ( i.e.,  any  effects  of 
manipulating  the  experimental  variables)  can  be  generalized  and  effectively  applied  by  others 
with  some  assurance  of  obtaining  the  same  or  at  least  similar  results.  Two  of  the  true 
experimental  designs  include  random  assignment  of  subjects  to  experimental  and  control 
groups  and  both  pre-  and  post-measurements  of  the  main  variable  of  interest  in  the 
experiment  (e.g.,  learning).  The  third  true  experimental  design  includes  the  random 
assignment  to  experimental  and  control  groups  but  omits  the  pre-measure.  The  most  common 
true  experimental  design  is  the  pretest-posttest  control  group  design. 

Experiments  that  do  not  involve  both  experimental  and  control  groups  or  do  not  involve 
random  assignment  to  experimental  or  control  groups  are  examples  of  what  Campbell  and 
Stanley  (1963)  refer  to  as  pre-experimental  designs  and  quasi-experimental  designs.  For 
example,  when  an  instructor  simply  changes  some  pedagogy  and  then  measures  students  to 
see  if  they  did  better  than  the  previous  class,  the  instructor  has  employed  a pre-experimental 
design.  This  design  is  called  the  one-shot  case  study  because  it  lacks  a control  group.  When 
an  instructor  uses  one  teaching  strategy  in  one  section  of  a course,  a different  strategy  in 
another  section  of  the  course,  and  then  compares  results  from  the  two  sections,  the  instructor 
has  employed  a quasi-experimental  design.  This  design  is  called  the  nonequivalent  control 
group  design  because  it  lacks  random  assignment  of  students  to  the  two  sections.  Although 
pre-experimental  and  quasi-experimental  designs  can  yield  useful  information  regarding 
effective  and  non-effective  instructional  strategies,  they  do  not  control  for  sources  of  internal 
invalidity  - that  is,  competing  hypotheses  remain  that  can  explain  the  “experimental”  results. 
These  are  not  true  experimental  designs. 

The  research  designs  employed  by  the  professors  involved  in  this  project  are  all  true 
experimental  designs.  All  the  designs  include  experimental  and  control  groups  (or  at  least 
two  distinct  experimental  groups,  each  as  a control  relative  to  the  other)  with  random 
assignment  and  both  pretesting  and  posttesting  of  the  students.  The  designs  are  all  essentially 
pretest-posttest  control  group  designs.  In  some  cases,  methodological  variations  were 
incorporated  to  fit  the  needs  of  the  classes  and  the  instructors  and  to  address  issues  of  fairness 
for  the  students,  such  as  alternating  the  experimental  intervention  between  the  two  groups  for 
balance.  The  designs  remain  true  experimental  designs,  and  their  results  should  be 
generalizable  to  contexts  similar  to  those  in  these  professors’  classrooms  with  similar 
operationalized  variables  and  similar  measurement  procedures. 

Wankat  et  al.  also  suggest  that  one  of  the  obstacles  to  the  use  of  true  experimental  designs  in 
engineering  classes  is  that  few  classes  “have  enough  students  to  form  experimental  and 
control  groups  large  enough  to  yield  statistically  significant  results”  (as  cited  in  Huber  & 
Morreale,  2002).  Although  most  classes  do  not  have  large  numbers  of  students  that  could 
yield  relatively  large  experimental  and  control  groups,  small  classes  do  not  prevent  true 
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experimental  research  nor  do  they  reduce  the  validity  of  the  statistical  analysis.  A class  as 
small  as  fifteen  or  twenty  students,  or  even  smaller,  can  probably  be  randomly  split  into  two 
groups,  and  the  application  of  appropriate  statistical  analysis  procedures  can  result  in  valid 
statistical  results.  The  statistical  issue  between  large  groups  and  small  groups  is  that  a larger 
difference  between  the  experimental  and  control  groups  is  required  to  achieve  statistical 
significance  with  small  groups  than  with  large  groups.  The  results  from  large  groups  may, 
however,  have  a better  chance  of  being  similar  to  more  contexts  in  other  environments  and 
may  thus  be  more  generalizable  than  the  results  from  very  small  groups.  For  this  reason, 
when  presenting  experimental  results,  researchers  should  always  provide  some  descriptive 
characteristics  of  their  samples  to  allow  consumers  and  other  researchers  to  judge  the 
similarity  of  the  sample  to  their  own  environments. 

Classroom  Assessment  and  Research:  An  Update  on  Uses,  Approaches,  and  Research 
Findings  (1998)  connects  classroom  assessment,  classroom  research,  and  the  Scholarship  of 
Teaching  (Boyer,  1990).  Each  chapter  illuminates  the  importance  of  research  on  teaching  and 
learning  if  we  are,  in  fact,  going  to  improve  teaching  and  learning.  The  chapters  in  this  book 
cover  classroom  assessment  techniques  and  cognitive  learning  theory,  as  well  as  approaches, 
strategies,  and  specific  tools  for  both  research  and  assessment  in  classrooms.  One  chapter 
focuses  on  TQM,  while  others  are  on  cases,  performance  through  student  teams,  etc.  This 
volume  and  its  collection  of  articles  are  a great  resource  for  deepening  understanding  of  the 
Scholarship  of  Teaching  through  classroom  research  and  assessment,  but  especially  for 
understanding  the  difference  and  interdependent  relationship  between  the  research  and 
student  assessment.  Cross  discusses  the  shift  from  assessment  for  accountability  to 
assessment  for  improvement,  clearly  the  goal  in  research  on  teaching  and  learning  (as  cited  in 
Angelo,  1998).  Cross  advocates  for  involving  students  in  the  assessment  process  because 
students  can  inform  professors  about  how  effectively  they  are  learning  rather  than  getting 
feedback  too  late,  e.g.  after  a test.  He  suggests  using  a strategy  that  produces  a stream  of 
continual  feedback  on  how  well  learning  is  occurring.  The  minute  paper  discussed  elsewhere 
is  one  simple,  yet  engaging,  method.  Where  classroom  research  and  assessment  have  been 
used  interchangeably.  Cross  makes  the  distinction  between  the  two.  Classroom  assessment 
addresses  the  “what”  questions  of  teaching  and  student  learning:  what  is  going  on,  what  did 
students  learn  from  the  lesson,  what  did  they  not  understand,  and  what  do  they  have 
questions  about.  However,  classroom  research  addresses  understanding  the  “why”  and  “how” 
questions  regarding  learning.  Why  did  students  respond  in  a particular  way?  Why  do  they  not 
understand  the  content?  Why  are  they  having  difficulty?  Classroom  research  leads  us  to 
understand  how  students  learn  and  encourages  us  to  use  our  classrooms  as  laboratories  to 
study  teaching  and  learning. 

However,  very  few  professors  or  college  instructors  know  much  about  learning  or  its  process. 
Cross  goes  on  to  specify  a difference  between  traditional  research  and  classroom  research  (as 
cited  in  Angelo,  1998).  Classroom  research  is  not  an  add-on  activity.  It  must  be  embedded  in 
the  ongoing  work  of  teaching  and  student  learning  - the  work  of  the  class.  It  is  unlike 
research  in  the  disciplines  because  it  does  not  require  special  equipment  and  easy  access  to 
research  libraries  or  colleagues  engaged  in  cutting  edge  research.  Professors  and  instructors 
have  all  they  need  to  do  first-rate  research  right  in  their  classrooms.  Research  in  the 
classroom  should  involve  students  as  collaborators  rather  than  as  subjects.  This  pays  off  as 
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they  are  eager  to  participate  and  learn  a great  deal  about  their  own  learning  while 
simultaneously  learning  the  academic  skills  of  inquiry  and  analysis.  Very  important  to  note  is 
that  classroom  research  differs  from  traditional  research  in  that  it  completes  the  cycle  of 
asking  the  question  and  then  making  changes  in  teaching  practices  based  upon  the 
outcome(s)  of  the  research. 

The  long  held  traditional  research  practice  has  been  for  researchers  to  engage  in  a research 
project,  write  up  the  findings,  and  then  publish  the  results  with  recommendations  for  further 
research,  often  not  using  the  results  themselves.  This  is  a most  ineffective  design  if  one’s 
goal  is  to  improve  teaching  and  learning.  Teachers  are  too  busy  to  read  research  reports, 
especially  those  with  equivocal  findings  that  may  or  may  not  apply  to  their  situations. 
Professors  engaging  in  their  own  classroom  research  find  the  results  much  more  relevant  and 
useful,  while  still  contributing  to  the  body  of  research  knowledge  on  teaching  and  learning. 
Classroom  research  conducted  in  his/her  own  classroom  focuses  on  questions  that  the 
professor  finds  meaningful  and  relevant,  and  the  results  can  be  used  immediately.  Professors 
can  engage  in  using  what  they  learn  diagnostically  to  improve  teaching  and  learning.  They 
can  go  further  to  engage  students  in  “diagnostic  learning  logs,”  where  students  analyze  their 
own  learning  prompted  by  several  questions.  I have  used  this  approach  both  on  its  own 
throughout  classes  and  then  also  as  a method  tied  to  student  portfolio  development.  Also  in 
classroom  research  an  “N  of  one”  may  be  more  valuable  than  an  “N  of  hundreds”  to  assure 
statistical  significance  in  more  traditional  correlational  research.  Although  more  qualitative 
in  nature,  interviews,  logs,  and  other  methods  can  tell  a professor  much  more  than  statistics. 
However,  as  mentioned  above,  we  are  trying  to  use  good  quantitative  methods  and 
procedures  as  well  as  good  qualitative  methods  and  procedures,  as  the  use  of  both  often 
reveals  more.  The  best  research  scenario  is  when  professors  engage  in  research  on  teaching 
and  learning,  use  that  research  diagnostically  to  improve  teaching  and  learning,  and  follow 
through  with  publishing  their  results  so  they  inform  others  and  add  to  the  collective  larger 
body  of  knowledge. 

Cross  also  makes  another  important  point,  that  of  solitary  versus  community  engagement  (as 
cited  in  Angelo,  1998).  Whereas  research  has  often  been  solitary,  especially  in  engineering 
(isolated  splendor).  Cross  notes  that  once  faculty  begin  to  engage  in  classroom  research,  they 
become  eager  to  share  it  with  colleagues.  This  should  be  supported  formally  by  the 
administration,  as  classroom  research  can  become  isolated  as  well  if  there  is  no  easy 
mechanism  for  sharing  in  a community  environment.  This,  to  me,  is  as  important  as 
publishing  to  the  larger  external  community.  Research  questions  and  research  methods  can 
benefit  from  a community  process  where  professors  discuss,  question,  interpret,  draw 
conclusions,  and  seek  to  design  further  research.  Thus,  classroom  research  is  a great  stimulus 
for  the  creation  of  a faculty  learning  community  about  teaching  and  student  learning.  She 
advocates  for  Boyer’s  (1990)  position  that  college  teaching  is  a scholarly  profession  that  is 
coming  into  its  own.  She  further  notes  that  while  the  members  of  a profession,  engineering 
and  technology  in  our  case,  are  bound  together  by  sharing  their  knowledge  and  expertise  and 
by  making  judgments  about  the  practice  of  their  profession,  “strangely  missing  from  the 
profession  of  teaching  of  teaching  . . .is  the  ability  to  advance  the  profession  through  a shared 
base  of  knowledge  about  human  learning.  Classroom  research  has  the  potential  for 
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understanding  learning  well  enough  to  improve  it  - as  individual  classroom  teachers  and 
collective  faculties  dedicated  to  the  mission  of  improving  undergraduate  education”  (p.  12). 

Another  good  resource  to  provide  knowledge,  understanding,  and  tools  for  designing, 
executing,  and  analyzing  classroom  research  is  Classroom  Research  - Implementing  the 
Scholarship  of  Teaching  (Cross  & Steadman,  1996).  Although  there  are  many  others,  these 
two  resources,  and  more  if  one  chooses  to  simply  search  them  out,  provide  a good  foundation 
for  someone  interested  in  the  “grand  experiment”  of  good  teaching  through  the  Scholarship 
of  Teaching.  Others  that  resonate  for  our  initiative  are 

• The  Scholarship  of  Teaching  and  Learning:  A National  Initiative  (Cambridge,  2000) 
discusses  the  Pew  Scholars  Fellowship,  the  Campus  Program,  and  the  Work  with 
Scholarly  Societies  that  seek  to  support  Boyer’s  (1990)  call  for  action. 

• Advancing  the  Scholarship  of  Teaching  Through  Collaborative  Self-Study  (Louie  et 
ah,  2003)  describes  faculty  group  work  engaged  in  self-study  to  advance  scholarly 
inquiry  in  which  professors  examine  their  beliefs  and  actions  as  educators,  “allowing 
professors  to  renew  their  instructional  tools  as  well  as  discover  new  tools  to  convey 
the  rich  and  changing  knowledge  in  a discipline.”  Self  study,  compared  to 
participation  in  traditional  workshops,  brings  about  many  more  benefits  (e.g.,  focused 
learning,  real  contextual  focus  on  their  students  and  classrooms,  controlled  purpose 
and  outcomes,  tangible  and  relative  products,  and  more).  They  go  further  to  advocate 
self-study  research  in  which  university  faculty  members  build  upon  existing  research 
expertise  to  improve  their  teaching.  As  our  work  is  truly  collaborative  self-study,  we 
agree  with  these  authors  and  find  their  model  involving  assessment,  implementation, 
and  dissemination  affirming.  They  conclude,  as  we  have,  that  “collaborative  self- 
study  research  holds  significant  potential  for  creating  valid,  useful  pedagogical 
content  knowledge  as  well  as  for  improving  teaching  practice.”  (p.l51;  p.l64) 

• Teaching  as  an  Act  of  Scholarship  (Nkomo,  1996)  expresses  reflections  about  the 
connection  between  teaching  and  research  in  response  to  Nord’s  (1996)  essay. 

• Research/Teaching  Boundaries  (Nord,  1996)  reflects  on  the  relationship  between 
teaching  and  research. 

Action  Research  - Is  it  Different  and  Does  It  Count? 

Let  us  consider  another  hot  topic  in  today’s  educational  research  arena  - action  research. 
What  is  the  difference  between  the  Scholarship  of  Teaching  as  described  by  Boyer  (1990), 
the  quality  of  research  that  we  are  striving  for  in  this  initiative  (true  experimental  research 
with  control  groups,  etc.),  and  action  research.  Action  research  has  been  regarded 
“suspiciously”  by  some  and  has  not  been  acknowledged  as  worthy  research  methodology. 
However,  although  not  always  form,  not  usually  designed  with  control  groups,  or 
experimental  in  nature,  it  can  be  quite  beneficial.  Here  are  some  definitions: 

Action  research  entails  studying  your  own  situation  to  change  the  quality  of  processes 
and  results  within  it.  To  do  action  research  is  to  empower  yourself  to  study  your 
actions  so  that  your  future  actions  will  be  more  effective.  It  also  aims  to  improve  your 
professional  judgments  and  to  give  you  insight  into  how  better  to  achieve  your 
educational  goals.  Through  action  research,  you  can  convert  current  practices  into 
better  procedures,  better  instructional  strategies,  and  better  curriculum.  [It]  is  a 
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continuous  and  cyclical  professional  activity  that  is  integrated  into  your  regular 
practice.  (Schmuck,  2006,  p.28) 

This  certainly  reflects  the  discussion  by  Louie  et  al.  (2003)  mentioned  above. 

As  an  alternative  to  traditional  research,  action  research  is  practical,  participative, 
empowering,  interpretive,  tentative,  and  critical.  This  means  that  a professor  can  gain  insights 
from  data  that  lead  to  practical  changes,  and  that  a professor  and  his/her  students  and/or 
colleagues  can  work  together  to  collect  data  about  real  issues  rather  than  some  outside, 
disinterested  party  performing  the  study.  Engaging  together  creates  equal  participants  where 
together  a group  can  pool  its  perceptions,  attitudes,  and  interpretations  about  the  inquiry. 
There  are  no  right  or  wrong  answers  but  instead  a creation  of  awareness  of  new  possibilities 
and  directions  to  try  in  the  classroom,  and  finally,  there  is  critical  reflection  in  which  as  the 
group  as  “self-critical  change  agents”  explores  results,  feedback,  and  considers  them  in  light 
of  change  (Schmuck,  2006,  p.  29). 

Two  models  to  consider  are  proactive  and  responsive  action  research.  Critical  reflection 
(Reflective  Practice)  might  lead  one  to  try  a new  practice  in  which  one  wanted  to  better 
connect  values  with  practice.  Steps  in  this  proactive  action  research  process  are 

1 . List  hopes  and  concerns  for  the  new  practice. 

2.  Perform  a knowledge  search  and  then  try  the  new  practice  for  more  effectiveness 
and  better  outcomes. 

3.  Search  for  the  best  research  procedures;  collect  data  to  track  student  reactions  and 
behavioral  changes. 

4.  Analyze  the  data. 

5.  Reflect  on  alternative  ways  to  behave. 

6.  Fine-tune  the  new  practice. . .or  try  a different  new  practice.  (Schmuck,  2006, 
p.32) 

With  responsive  action  research,  diagnostic  data  is  collected  before  trying  a new  practice; 
therefore,  one  engaging  in  the  responsive  process  would 

1 . After  searching  for  appropriate  methods,  collect  data  to  diagnose  the  situation. 

2.  Analyze  the  data  for  patterns,  themes,  or  ideas  for  action  or  practice. 

3.  Perform  a knowledge  search,  distribute  the  data,  and  announce  the  changes  that 
will  be  made  [to  the  students] . 

4.  List  [for  them]  what  is  desired  - the  hopes  or  concerns  for  the  new  practice. 

5.  Try  the  new  practice  for  more  effectiveness  and  better  outcomes. 

6.  Collect  data  and  evaluate  the  new  practice. 

The  phases  of  action  research  could  be  considered  Initiation,  Detection,  and  Judgment. 
Schmuck  (2006)  provides  designs,  methods,  tools,  and  good  discussion  about  action 
research.  It  differs  in  that  a researcher  will  not  use  true  experimental  designs;  action  research 
comes  closer  to  Campbell  and  Stanley’s  (1963)  pre-experimental  or  quasi-experimental 
designs,  where  there  are  no  control  groups  or  random  assignment  of  students  into  groups  for 
comparison. 
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For  example,  when  an  instructor  simply  changes  some  pedagogy  and  then  measures  students 
to  see  if  they  did  better  than  the  previous  class,  the  instructor  has  employed  a pre- 
experimental  design;  this  design  is  called  the  one-shot  case  study  because  it  lacks  a control 
group.  When  an  instructor  uses  one  teaching  strategy  in  one  section  of  a course  and  a 
different  strategy  in  another  section  of  the  course  and  then  compares  results  from  the  two 
sections,  the  instructor  has  employed  a quasi-experimental  design;  this  design  is  called  the 
nonequivalent  control  group  design  because  it  lacks  random  assignment  of  students  to  the 
two  sections.  Although  pre-experimental  and  quasi-experimental  designs  can  yield  useful 
information  regarding  effective  and  non-effective  instructional  strategies,  they  do  not  control 
for  sources  of  internal  invalidity  - that  is,  there  remain  competing  hypotheses  that  can 
explain  the  “experimental”  results.  These  are  not  true  experimental  designs. 

However,  do  not  underestimate  the  relevance,  significance,  or  value  of  Action  Research, 
even  with  its  constraints  or  limitations.  When  well  researched,  thought  out  and  formulated, 
and  where  appropriate  consideration  is  given  to  the  design  and  execution  of  the  research, 
where  the  new  practice  or  strategy  is  well  defined  and  described,  and  especially  where  the 
student  assessments  have  high  integrity,  action  research,  to  me,  is  one  type  of  research  that  is 
valuable  and  should  be  considered  an  accompanying  option  in  the  Scholarship  of  Teaching. 

In  my  humble  opinion,  there  should  be  a mix,  ongoing  action  research  combined  with  formal 
true  experimental  and  quasi-experimental  research. 

The  Scholarship  of  Teaching,  whether  in  its  highest  form  (experimental),  very  strong  forms 
(pre-experimental  and  quasi-experimental),  or  somewhat  more  informal  or  less  controlled 
form  (action  research),  all  have  the  same  goal  - improving  teaching  for  higher  or  increased 
student  learning;  therefore,  each  deserves  its  respective  place  in  a professor’s  research 
repertoire  and  acknowledgment  of  its  potential  to  contribute  to  the  goal  of  adding  knowledge 
to  the  fields  of  teaching  and  student  learning.  A few  other  good  resources  on  action  research 
are 

Action  Research  -Teachers  as  Researchers  in  the  Classroom  (2006) 

Macintyre.  (2000).  The  Art  of  Action  Research  in  the  Classroom 
McNiff  & Whitehead.  (2002).  Action  Research 
Mills.  (2000).  Action  Research 

Noffke  & Stevenson.  (1995).  Educational  Action  Research 

Somekh.  (2006).  Action  Research  - a methodology  for  change  and  development 

Zuber-Skerritt.  (1992).  Action  Research  in  Higher  Education 

Program  and  Classroom  Research  Designs 

As  mentioned  in  the  faculty  development  program  section,  this  project  consists  of  faculty 
development  in  preparation  for  classroom  research,  qualified  by  Boyer  (1990)  as  the 
Scholarship  of  Teaching.  The  participating  professors  are  from  the  College  of  Engineering 
and  Engineering  Technology  (CEET)  at  Northern  Illinois  University.  The  professors  were 
selected  for  participation  in  the  program  by  the  college  administration.  The  faculty 
development  program  included  traditional  and  performance-based  faculty  development  in  the 
areas  of  reflective  practice  - course  and  teaching  analysis;  course  development,  testing,  and 
performance  assessment/rubric  analysis  and  development;  teaching  and  learning  strategies; 
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and  research  design,  methods,  and  procedures.  Participants  were  assessed  traditionally  and 
through  performances.  The  professors  participated  in  the  development  activities  during  the 
spring  of  2006.  (See  the  Faculty  Development  Program  section  of  this  document  - A. 5). 

The  research  component  of  the  program  includes  experimental  studies  by  the 
professors  in  their  own  classrooms  that  are  intended  to  provide  the  professors  with 
authentic,  real  world,  educational  research  experiences.  The  experimental  conditions 
of  these  studies  involved  the  application  of  different  models,  strategies,  and 
procedures  learned  about  during  the  faculty  development  program  Spring  2006.  For 
example,  some  professors  examined  the  effects  of  traditional  versus  performance 
assessment  in  their  classrooms,  while  other  professors  examined  the  effects  of 
different  teaching  methods  - passive  versus  active  learning  and  different  levels  of 
cooperative  learning  techniques. 

All  the  professors  received  training  on  all  faculty  development  program  components 
in  Spring  2006;  however,  the  individual  classroom  implementation  and  research 
varied  across  professors.  The  following  designs  explain  their  choices  in  teaching 
strategies/models  and/or  student  assessment  procedures.  Because  all  professors 
implemented  changes  in  their  courses  and  syllabus  as  a result  of  their  analyses  and 
because  they  all  developed  new  traditional  tests  and  new  performance  assessments, 
these  components  were  implemented  by  all  participating  professors  during  the  2006 
research  semester.  However,  because  of  the  integral  nature  of  each  of  these 
components  relative  to  each  other,  the  effectiveness  of  these  components  will  be 
assessed  as  a single  unit.  That  is,  the  effectiveness  of  course  and  teaching  analyses 
will  not  be  isolated  from  the  effectiveness  of  classroom  testing  and  assessment, 
except  where  professors  have  chosen  to  research  the  use  of  traditional  tests  versus 
performance  assessments.  For  a brief  and  broad  review,  see  the  Faculty  Development 
Program  Section  A. 5: 

Program  Component  1:  Reflective  Practice  - Course  and  Teaching  Analyses 
Program  Component  2:  Student  Assessment  - Traditional  Tests 
Program  Component  3:  Performance  Assessment  - Performance  Tasks 
and  Rubrics 

Program  Component  4:  Teaching  Strategies,  Models,  Methods,  Techniques 
Program  Component  5:  Classroom  Research 

The  effectiveness  of  the  overall  Faculty  Development  Program  was  evaluated  through 
examination  of  several  assessment  variables.  The  variables  are  briefly  described 
below.  Also  included  with  these  descriptions  are  statements  indicating  which 
program  components  are  evaluated  by  each  variable.  The  relationships  between  the 
program  components  and  the  assessment  variables  are  also  presented  in  Table  1. 

1 . Self  Perceptions  of  Competency;  The  professors  were  asked  to  complete  a 
questionnaire  soliciting  their  perceptions  of  their  competency  levels  with 
respect  to  different  aspects  of  the  faculty  development  program.  Indices  of 
competency  scores  were  developed  and  analyzed  statistically.  The 
assessment  of  the  professors’  perceptions  of  competency  addressed  the 
overall  effectiveness  of  the  faculty  development  program. 
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2.  Content  Knowledge  of  Specific  Components.  Component- specific  content 
knowledge  was  assessed  through  a questionnaire  or  quiz  administered 
before  the  delivery  of  each  program  component  and  at  two  different  times 
after  the  component  development.  The  assessment  of  the  professors’ 
content  knowledge  for  each  component  addressed  the  effectiveness  of  the 
content  development  for  each  separate  component. 

3.  Value/Usefulness  of  Specific  Components.  Brief  evaluations  of  each 
component  were  obtained  after  the  development  activities  for  each 
program  component.  Questions  such  as  “Was  the  information  useful?”  and 
“Would  you  recommend  that  other  faculty  learn  the  material?”  were 
asked.  These  evaluations  addressed  the  value  and  usefulness  of  each 
separate  component. 

4.  End  of  Course  Evaluations  - A)  NIU  (current)  B)  Project  (new).  In 
addition  to  the  current  standard  course  evaluations  that  students  provide 
for  each  class,  a separate  end-of-course  evaluation  instrument  was 
developed  to  assess  specific  aspects  covered  in  the  faculty  development 
program.  The  evaluations,  administered  in  the  professors’  classes  both 
before  and  after  the  delivery  of  the  faculty  development  program 
examined  whether  the  professors  were  successful  in  implementing  the 
content  of  the  program  in  their  classrooms  - based  on  students’ 
perceptions. 

5.  Test  Analysis.  One  method  for  improving  an  existing  test  is  to  perform  a 
test  analysis.  The  quantitative  procedures  of  test  analysis  will  result  in  the 
identification  of  specific  items  that  do  not  “perform”  as  expected  by  the 
instructor:  for  example,  an  item  may  be  too  difficult,  or  perhaps  some  of 
the  more  proficient  students  answer  an  item  incorrectly,  while  less 
proficient  students  answer  it  correctly.  To  assess  the  degree  to  which  the 
modified  test  has  actually  been  improved,  the  proportion  of  poorly 
performing  items  from  the  modified  posttest  (e.g.,  the  Eall  2006  final)  will 
be  compared  to  the  proportion  of  poorly  performing  items  from  the 
original  pretest  (e.g.,  the  Eall  2005  final). 

6.  Performance  Assessments.  Although  no  performance  assessments  were 
administered  for  Eall  2005,  all  professors  developed  three  complex 
performance  tasks  with  corresponding  rubrics.  These  were  administered 
by  each  professor  for  Eall  2006.  The  results  of  using  performance 
assessment  as  an  additional  type  of  student  learning  assessment  will  be 
judged  qualitatively  from  professor  self-reports  about  their  reactions  to 
performance  assessment  as  well  as  any  information  stemming  from 
student  reactions. 

7.  Course  Grades.  Due  to  the  number  and  variety  of  factors  that  can  affect  course 
grades,  confounding  year-to-year  comparisons,  no  analyses  involving  grades  were 
performed.  It  is  suggested  here,  as  well  as  with  course  grades,  that  professors 
include  course  grades  as  a factor  in  the  research  design. 
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8.  Portfolios.  The  professors  built  formal  teaching  portfolios  that  provide  full 
evidence  of  their  new  knowledge  and  skills  on  teaching  and  learning.  These 
provided  the  opportunity  to  judge  the  quality  of  their  analyses,  level  of  learning, 
and  educational  products  to  be  used  in  the  classroom. 

9.  Research.  The  professors  have  engaged  only  in  engineering-related  research  up 
until  now;  there  had  been  no  educational  research.  Student  learning  data  were 
collected  and  analyzed.  Research  results  were  reviewed  in  manuscripts  prepared 
by  each  professor  for  publication  during  Spring  2007. 
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Table  A.3.1  presents  the  relationships  between  the  five  program  components  and  the 
seven  assessment  variables. 


Table  A.3.1:  Assessment  Variables  Used  to  Evaluate  Different 
Training  Program  Components 


Faculty  Development  Program  Components 


Assessment 

Variables 

1. 

Analysis 

2. 

Testing 

3. 

Performance 

4. 

Methods 

5. 

Research 

1.  Self 

Perceptions 

of 

Competency 

To  evaluate  the  overall  training  program 

2.  Content 
Knowledge 
of  Specific 
Components 

Component 

1 

Assessment 

Component 

2 

Assessment 

Component 

3 

Assessment 

Component 

4 

Assessment 

Component 

5 

Assessment 

3.  Program 
Feedback 
and 

Evaluation 

Component 

1 

Assessment 

Component 

2 

Assessment 

Component 

3 

Assessment 

Component 

4 

Assessment 

Component 

5 

Assessment 

4.  End  of 
Course 
Evaluations: 

A) 

NlU(current) 

B) 

Project(new) 

To  evaluate  components  1-4,  overall 

X 

5.  Test  Analysis 

To  evaluate  the 
combination  of  components 
1 &2 

6.  Performance 
Assessments 

To  evaluate  components  1,  2,  & 3 

sx 

7.  Course 
Grades 

Due  to  the  number  and  variety  of  factors  that  can  affect  course  grades, 
confounding  year-to-year  comparisons,  no  analyses  involving  grades  was 

performed. 

*8.  Professors’ 
Portfolios 

To  evaluate  the  overall  faculty  development  program 

*9.  Professors’ 
Research 

To  evaluate  the  overall  faculty  development  program 
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The  complete  matrix  of  data  sources,  times  for  pre-  and  post-  data  collection,  and  the 
specific  statistical  analysis  for  each  of  the  seven  assessment  variables  are  presented  in 
Table  A.3.2. 


Table  A.3.2:  Data  Sources,  Measurement  Times,  and  Analysis  Procedures 

for  Each  of  the  Assessment  Variables 


Assessment 

Variables 

Data  Source 

Pre 

Measures 

Post  1 
Measures 

Post  2 
Measures 

Analysis 

1.  Self 

Perceptions  of 
Competency 

Professors 

At  Program 
Orientation 

After 

Overall 

Training 

End  of  Eall 
2006 
Courses 

Dependent 
Groups  t-tests 

2.  Content 
Knowledge  of 
Specific 
Components 

Professors 

Prior  to 
Component 
Training 

After  Each 
Component 
is  Presented 

End  of  Eall 
2006 
Courses 

Dependent 
Groups  t-tests 

3.  Program 
Feedback  and 
Evaluation 

Professors 

After  Each 
Component 
is  Presented 

Qualitative 

4.  End  of  Course 
Evaluations: 

A) NIU 
(current) 

B) Project 
(new) 

Students  in 
Professors’ 
Classes 

End  of  Eall 
2005 
Courses 

End  of  Eall 
2006 
Courses 

Independent 
Groups  t-tests 

5.  Test  Analysis 

Students  in 
Professors’ 
Classes 

Eall  2005 
Courses 

Eall  2006 
Courses 

Independent 
Groups  t-tests 

6.  Performance 
Assessments 

Students  in 
Professors’ 
Classes 

Eall  2006 
Courses 

Qualitative  / 
Correlations 

7.  Course 
Grades 

Due  to  the  number  and  variety  of  factors  that  can  affect  course  grades, 
confounding  year-to-year  comparisons,  no  analyses  involving  grades  was 

performed. 

8.  Professors’ 
Portfolios 

Professors 

After 

Overall 

Program 

Completion 

Qualitative 

9.  Professors’ 
Research 

Professors 

Spring  2007 
and  Beyond 

Qualitative 

The  first  post  measurement  for  variable  1 will  occur  shortly  after  the  delivery  of  the  entire 
training  program;  for  variables  2 and  3,  the  first  post  measure  occurs  after  the  delivery  of  each 
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separate  component.  This  first  post  measurement  will  address  the  immediate  effectiveness  of 
the  program  delivery  and  the  value  and  usefulness  of  the  content.  The  second  measurement  of 
variables  1 and  2 at  the  end  of  the  Fall  2006  courses  will  address  the  stability  (the  retention) 
of  the  knowledge  gained  during  the  program  delivery  phase  and  after  the  implementation  and 
reinforcement  of  that  knowledge  during  the  Fall  2006  term. 

The  data  sources  for  variables  4-7  are  the  students  in  the  professors’  classes  for  Fall 
2005  (pre)  and  Fall  2006  (post).  Because  these  are  distinct  (independent)  groups,  it  is 
appropriate  to  analyze  the  data  through  independent  group  t-tests,  except  where 
qualitative  analyses  are  indicated. 

* The  assessment  variables  8 and  9 will  be  reviewed  and  judged  qualitatively  as  to 
their  effect  on  the  overall  faculty  development  program.  Quantitative  student  learning 
data  will  be  collected  during  assessment  of  variable  9.  Although  student  gain  could  be 
considered  an  evaluative  measure  of  the  overall  faculty  development  program,  a 
second  measure  is  also  inherent  in  that  variable:  professors  performing  formal 
research  on  teaching  and  learning.  Therefore,  there  will  be  a more  qualitative 
approach  to  this  analysis. 

The  specific  research  studies  to  be  performed  by  the  professors  are  discussed  below. 
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Regina  Rahn  - Primary  Study 

Model  1 -Traditional  Test  with  Performance  Assessment 


This  study  is  designed  to  examine  the  following  questions: 

1.  Does  the  administration  of  a performance  assessment  that  covers  a subset 
of  content  also  covered  in  a traditional  midterm  result  in  increased 
learning  beyond  the  administration  of  the  traditional  midterm  alone  as 
indicated  by  the  scores  on  the  traditional  midterm? 

2.  Does  the  administration  of  a performance  assessment  that  covers  a subset 
of  content  also  covered  in  a traditional  midterm  result  in  increased 
knowledge  retention  beyond  the  administration  of  the  traditional  midterm 
alone  as  indicated  by  a separate  final  exam? 

The  basic  design  for  this  study  is  presented  in  the  table  below. 


Table  A. 3. 3:  Model  1 

Model  1:  Traditional  Test  With  Performance  Assessment 


Treatment 

Posttest  1 

Posttest  2 

Experimental 

Group 

Instruction 

Performance 
Assessment 
Related  to 
Traditional  Test 

Traditional 

Midterm 

-> 

Traditional 

Final 

Control 

Group 

Instruction 

Performance 
Assessment  Not 
Related  to 
Traditional  Test 
(Placebo) 

Traditional 

Midterm 

Traditional 

Final 

Students  will  be  randomly  assigned  to  the  experimental  and  control  groups,  and  therefore, 
those  groups  can  be  considered  equivalent.  The  treatment  for  the  experimental  group  is 
essentially  the  administration  of  the  traditional  midterm  along  with  some  performance 
assessment  activities  covering  some  of  the  same  content  that  is  covered  in  the  traditional 
midterm.  The  treatment  for  the  control  group  is  the  administration  of  the  traditional  midterm 
and  a placebo  - perhaps  some  performance  assessment  activities  that  are  not  related  to  the 
content  in  the  traditional  midterm.  After  the  administration  of  the  Traditional  Test  (Posttest  1), 
the  distinction  between  the  groups  is  dissolved  with  instruction  and  assessment  activities 
delivered  to  all  students  equally. 

• Question  1 will  be  addressed  by  comparing  the  traditional  midterm  means  across  the 
experimental  and  control  groups. 

• Question  2 will  be  addressed  by  comparing  the  final  exam  means  across  the 
experimental  and  control  groups  (for  the  content  in  the  final  that  is  also  in  the 
midterm)  across  the  two  groups. 
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Regina  Rahn  — Secondary 

Model  4 - Individual  Learning  versus  Cooperative  Learning 

This  study  is  designed  to  examine  the  following  questions: 

1 . Does  individual  learning  versus  cooperative  learning  result  in  differential 
knowledge  gains  as  indicated  by  a traditional  cognitive  test? 

2.  Does  individual  learning  versus  cooperative  learning  result  in  differential 
knowledge  retention  as  indicated  by  a final  exam? 

The  design  for  this  study  is  presented  in  the  table  below. 


Table  A. 3.4:  Model  4 


Model  4:  Individual  Learning  vs.  Cooperative  Learning 

Treatment 

Posttest  1 

Posttest  2 

Individual 

Learning 

Group 

Individual 

Learning 

Traditional  Test 

Final 

Cooperative 

Learning 

Group 

Cooperative 

Learning 

Traditional  Test 

Final 

Treatment 

Group 

Content  Area 
I 

Content  Area 
II 

Content  Area 
III 

Content  Area 
IV 

1 

Individual 

Cooperative 

Individual 

Cooperative 

2 

Cooperative 

Individual 

Cooperative 

Individual 

Random  assignment  of  students  to  two  groups  will  allow  us  to  assume  the  groups  are 
equivalent.  The  actual  delivery  of  the  treatment  conditions  will  alternate  across  content 
areas  and  groups,  as  shown  in  the  blowout  diagram.  Group  1 will  be  the  individual  learning 
group  for  content  areas  I and  III,  while  group  2 will  be  the  individual  learning  group  for 
content  areas  II  and  IV.  This  will  add  to  the  validity  of  the  design  and  enhance  the  fairness 
of  the  treatment  conditions  within  the  student  groups.  For  this  delivery  model  to  work,  there 
needs  to  be  an  even  number  of  content  areas  with  a minimum  of  two.  For  fairness  to 
students,  each  content  area  should  also  be  weighted  approximately  equally  within  both 
posttests.  After  the  administration  of  the  Traditional  Test  (Posttest  1),  the  distinction 
between  the  groups  is  dissolved  with  instruction  and  assessment  activities  delivered  to  all 
students  equally. 

• Question  1 will  be  addressed  by  comparing  the  posttest  1 means  under  the 
individual  and  cooperative  learning  conditions. 

• Question  2 will  be  addressed  with  similar  comparisons  on  the  posttest  2 
means. 


15 


Abul  Azad 


Model  2 - Performance  Assessment  and  Traditional  Test  Administered  in  Different  Order 


This  study  is  designed  to  examine  the  following  questions: 

1 . Does  a performance  assessment  administered  in  conjunction  with  a traditional  test 
result  in  increased  learning  beyond  the  traditional  test  alone  as  indicated  by  scores 
on  the  traditional  test? 

2.  Does  the  order  of  the  administration  of  a performance  assessment  and  a traditional 
test  result  in  differential  learning  as  indicated  by  the  combination  of  the 
performance  assessment  and  traditional  test  scores? 

3.  Does  the  order  of  administration  of  a performance  assessment  and  a traditional  test 
affect  knowledge  retention  as  indicated  by  a final  exam? 

The  basic  design  for  this  study  is  presented  in  the  table  below. 

Table  A. 3. 5:  Model  2 

Model  2:  Performance  Assessment  and  Traditional  Test 
Administered  in  Different  Order 


Group  1 

Instruction 

Performance 

Assessment 

Traditional  Test 

Final 

Group  2 

Instruction 

Traditional  Test 

Performance 

Assessment 

Final 

Students  will  be  randomly  assigned  to  the  experimental  and  control  groups,  and  therefore, 
those  groups  can  be  considered  equivalent.  Group  1 will  receive  the  performance  assessment 
before  the  traditional  test,  and  group  2 will  receive  the  traditional  test  before  the  performance 
assessment. 

• Question  1 will  be  addressed  by  comparing  the  traditional  test  means  between  the 
two  groups. 

• Question  2 will  be  addressed  by  comparing  the  combined  performance  assessment 
and  traditional  test  scores  between  the  two  groups. 

• Question  3 will  be  addressed  by  comparing  scores  from  the  final  that  are  based  on 
material  also  covered  on  the  midterm  (the  performance  assessment  and  the 
traditional  midterm  test)  between  the  two  groups. 
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Abhijit  Gupta 

Model  2 - Performance  Assessment  and  Traditional  Test  Administered  in  Different  Order 


This  study  is  designed  to  examine  the  following  questions: 

1 . Does  a performance  assessment  administered  in  conjunction  with  a traditional  test 
result  in  increased  learning  beyond  the  traditional  test  alone  as  indicated  by  scores 
on  the  traditional  test? 

2.  Does  the  order  of  the  administration  of  a performance  assessment  and  a traditional 
test  result  in  differential  learning  as  indicated  by  the  combination  of  the 
performance  assessment  and  traditional  test  scores? 

3.  Does  the  order  of  administration  of  a performance  assessment  and  a traditional  test 
affect  knowledge  retention  as  indicated  by  a final  exam? 

The  basic  design  for  this  study  is  presented  in  the  table  below. 


Table  A. 3. 6:  Model  2 

Model  2:  Performance  Assessment  and  Traditional  Assessment 
Administered  in  Different  Order 


Group  1 

Instruction 

Performance 

Assessment 

Traditional 

Test 

Final 

Group  2 

Instruction 

Traditional  Test 

Performance 

Assessment 

Final 

Students  will  be  randomly  assigned  to  the  experimental  and  control  groups,  and  therefore, 
those  groups  can  be  considered  equivalent.  Group  1 will  receive  the  Performance  assessment 
before  the  traditional  test,  and  group  2 will  receive  the  traditional  test  before  the  Performance 
assessment. 

• Question  1 will  be  addressed  by  comparing  the  traditional  test  means  between  the 
two  groups. 

• Question  2 will  be  addressed  by  comparing  the  combined  performance  assessment 
and  traditional  test  scores  between  the  two  groups. 

• Question  3 will  be  addressed  by  comparing  scores  from  the  final  that  are  based  on 
material  also  covered  on  the  midterm  (the  performance  assessment  and  the 
traditional  midterm  test)  between  the  two  groups. 
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Reinaldo  Moraga 

Model  4 - Individual  Learning  versus  Cooperative  Learning 

This  study  is  designed  to  examine  the  following  questions: 

1 . Does  individual  learning  versus  cooperative  learning  result  in  differential 
knowledge  gains  as  indicated  by  a traditional  test? 

2.  Does  individual  learning  versus  cooperative  learning  result  in  differential 
knowledge  retention  as  indicated  by  a final  exam? 

The  design  for  this  study  is  presented  in  the  table  below. 


Table  A.3.7:  Model  4 

Model  4:  Individual  Learning  vs.  Cooperative  Learning 


Treatment 

Posttest  1 

Posttest  2 

Individual 

Learning 

Group 

Individual 

Learning 

Traditional  Test 

Final 

Cooperative 

Learning 

Group 

Cooperative 

Learning 

Traditional  Test 

Final 

Treatment 

Group 

Content  Area 
I 

Content  Area 
II 

Content  Area 
III 

Content  Area  IV 

1 

Individual 

Cooperative 

Individual 

Cooperative 

2 

Cooperative 

Individual 

Cooperative 

Individual 

Random  assignment  of  students  to  two  groups  will  allow  us  to  assume  the  groups  are 
equivalent.  The  actual  delivery  of  the  treatment  conditions  will  alternate  across  content  areas 
and  groups,  as  shown  in  the  blowout  diagram.  Group  1 will  be  the  individual  learning  group 
for  content  areas  I and  III,  while  group  2 will  be  the  individual  learning  group  for  content 
areas  II  and  IV.  This  will  add  to  the  validity  of  the  design  and  enhance  the  fairness  of  the 
treatment  conditions  within  the  student  groups.  For  this  delivery  model  to  work,  there  needs 
to  be  an  even  number  of  content  areas  with  a minimum  of  two.  For  fairness  to  students,  each 
content  area  should  also  be  weighted  approximately  equally  within  both  posttests.  After  the 
administration  of  the  Traditional  Test  (Posttest  1),  the  distinction  between  the  groups  is 
dissolved  with  instruction  and  assessment  activities  delivered  to  all  students  equally. 

• Question  1 will  be  addressed  by  comparing  the  posttest  1 means  under  the 
individual  and  cooperative  learning  conditions. 

• Question  2 will  be  addressed  with  similar  comparisons  on  the  posttest  2 
means. 
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Robert  Tatara 


Model  4 - Individual  Learning  versus  Cooperative  Learning 

This  study  is  designed  to  examine  the  following  questions: 

1 . Does  individual  learning  versus  cooperative  learning  result  in  differential 
knowledge  gains  as  indicated  by  a traditional  test? 

2.  Does  individual  learning  versus  cooperative  learning  result  in  differential 
knowledge  retention  as  indicated  by  a final  exam? 

The  design  for  this  study  is  presented  in  the  table  below. 


Table  A. 3. 8:  Model  4 

Model  4:  Individual  Learning  vs.  Cooperative  Learning 


Treatment 

Posttest  1 

Posttest  2 

Individual 

Learning 

Group 

Individual 

Learning 

Traditional  Test 

Final 

Cooperative 

Learning 

Group 

Cooperative 

Learning 

Traditional  Test 

Final 

Treatment 


Group 

Content  Area 
I 

Content  Area 
II 

Content  Area 
III 

Content  Area 
IV 

1 

Individual 

Cooperative 

Individual 

Cooperative 

2 

Cooperative 

Individual 

Cooperative 

Individual 

Random  assignment  of  students  to  two  groups  will  allow  us  to  assume  the  groups  are 
equivalent.  The  actual  delivery  of  the  treatment  conditions  will  alternate  across  content  areas 
and  groups,  as  shown  in  the  blowout  diagram.  Group  1 will  be  the  individual  learning  group 
for  content  areas  I and  III,  while  group  2 will  be  the  individual  learning  group  for  content 
areas  II  and  IV.  This  will  add  to  the  validity  of  the  design  and  enhance  the  fairness  of  the 
treatment  conditions  within  the  student  groups.  For  this  delivery  model  to  work,  there  needs 
to  be  an  even  number  of  content  areas  with  a minimum  of  two.  For  fairness  to  students,  each 
content  area  should  also  be  weighted  approximately  equally  within  both  posttests.  After  the 
administration  of  the  Traditional  Test  (Posttest  1),  the  distinction  between  the  groups  is 
dissolved  with  instruction  and  assessment  activities  delivered  to  all  students  equally. 

• Question  1 will  be  addressed  by  comparing  the  posttest  1 means  under  the 
individual  and  cooperative  learning  conditions. 

• Question  2 will  be  addressed  with  similar  comparisons  on  the  posttest  2 
means. 
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Ibrahim  Abdel-Motaleb 


Custom  Model:  Individual  Performance-Based  Learning  versus  Cooperative  Performance- 
Based  Learning 

The  basic  research  question  in  this  model  is 

1 . Does  individual  performance-based  learning  or  cooperative  performance- 
based  learning  result  in  better  student  learning  as  reflected  in  an  end-of- 
unit  exam  (either  midterm  or  final). 


Table  A. 3.9:  Research  Model 


Research  Model:  Individual  vs.  Cooperative  Performance-Based  Learning 

Treatment 

Posttest 

Treatment 

Posttest 

Experimental 
Group  1 

Instruction 

Individual 

Performance- 

Based 

Midterm 

Instruction 

Cooperative 

Performance- 

Based 

Pinal 

Experimental 
Group  2 

Instruction 

Cooperative 

Performance- 

Based 

Midterm 

Instruction 

Individual 

Performance- 

Based 

Pinal 

Replication  1 

Replication  2 

Students  will  be  randomly  assigned  to  the  two  experimental  groups,  and  therefore,  those 
groups  can  be  considered  equivalent.  The  experiment  will  actually  be  administered  twice 
during  the  semester.  To  enhance  the  fairness  to  the  class,  both  groups  will  be  exposed  to  both 
treatment  conditions  - alternating  across  the  two  replications.  The  final  is  not  a 
comprehensive  final,  so  there  should  be  no  carryover  from  the  midterm  to  the  final  that  might 
contaminate  the  interpretation  of  the  results. 

The  research  question  will  be  addressed  by  comparing  the  mean  test  scores  between 
the  two  groups  for  each  replication  of  the  experiment. 
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Brianno  Coller 


Model  1:  Hands-On  Manipulative  Procedures  versus  Graphical  Procedures 

This  study  is  designed  to  examine  two  questions: 

1.  In  problem-based-leaming  projects,  are  hands-on  manipulative  procedures 
more  effective  on  learning  than  graphical  procedures,  as  reflected  in  a 
midterm  exam? 

2.  In  problem-based-leaming  projects,  are  hands-on  manipulative  procedures 
more  effective  on  retention  of  learning  than  graphical  procedures,  as 
reflected  in  a final  exam? 

Figure  A.3.10:  Model  1 


Model  1:  Hands-On  Manipulative  vs.  Graphical 

Treatment 

Posttest  1 

Posttest  2 

Experimental 
Group  1 

Instruction 

Hands-On 

Manipulative 

Midterm 

Pinal 

Experimental 
Group  2 

Instruction 

Graphical 

Midterm 

-> 

Pinal 

Students  will  be  randomly  assigned  to  the  two  experimental  groups,  and  therefore, 
those  groups  can  be  considered  equivalent.  Each  group  will  be  given  some  problem- 
based  learning  tasks.  Group  1 will  attempt  to  resolve/complete  the  tasks  with  hands-on 
manipulation  of  physical  objects,  while  group  2 will  attempt  to  resolve/complete  the 
problems  with  graphical  techniques.  After  the  administration  of  the  midterm  (posttest 
1),  the  distinction  between  the  groups  is  dissolved  with  instmction  and  assessment 
activities  delivered  to  all  students  equally. 

• Question  1 will  be  addressed  by  comparing  the  posttest  1 means 
between  the  two  groups. 

• Question  2 will  be  addressed  by  comparing  the  posttest  2 means 
between  the  groups  and  by  comparing  the  posttest  1 means  to  the 
posttest  2 means  (for  the  items  in  the  final  that  cover  material  in  the 
midterm)  across  the  two  groups. 
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Results  and  Conclusions 


Results 

The  research  results  can  be  reviewed  from  two  perspectives:  the  program  and  classroom 
research. 

Regarding  the  faculty  development  program: 

The  faculty  development  program  was  very  successful  across  all  its  components.  The 
participating  faculty  member  feedback  was  excellent,  and  their  educational  products  were  all 
developed  to  quality  expectations.  In  fact,  considering  that  this  was  the  first  time  faculty 
members  had  analyzed  their  courses  formally  (and  to  such  high  standards);  the  first  time  that 
they  had  considered  teaching  practices,  or  engaged  in  intense  critical  reflection,  about 
teaching  models,  styles,  learning  styles.  Bloom’s  Taxonomy,  and/or  Dale’s  Cone;  the  first 
time  they  developed  tests  and  performance  tasks/rubrics  to  particular  specifications  and  with 
some  knowledge  about  what  good  tests  and  performance  tasks/rubrics  are;  and  more,  their 
knowledge  and  skill  gain  and  their  educational  products  were  outstanding. 

Regarding  specific  program  assessment  variables 
There  was  significant  gain  on  the  part  of  all  faculty  members  in  their  self  perceptions  of 
competency  as  well  as  the  content  knowledge  of  program  components.  Faculty  members 
consistently  assigned  a very  high  value  to  the  usefulness  of  the  program  components  through 
the  feedback  process.  They  performed  well  according  to  their  performance  on  test  analysis 
procedures  and  the  development  of  their  new  objective  tests.  Also  they  produced  well 
developed  performance  tasks  and  rubrics  and  followed  through  with  formal  and  experimental 
classroom  research  during  their  redeveloped  courses,  using  their  new  educational  products, 
instructional  models,  styles,  strategies,  new  tests,  new  performance  tasks  and  rubrics  as  well 
as  selected  teaching  models,  styles.  Three  used  Kolb  or  Felder  learning  style  inventories,  and 
they  all  produced  research  manuscripts.  Finally,  each  professor  has  a teaching  portfolio 
reflecting  their  gain,  growth,  analysis  and  change  process,  and  educational  products. 

Faculty  members  suggested  shortening  the  program  for  sustainability;  however,  they  also 
requested  more  depth  on  cooperative  learning,  the  addition  of  programming  on  student 
teaming,  as  well  as  classroom  management  and  student  conflict  resolution  or  management. 
Finally,  there  would  be  a great  need  to  provide  additional  learning  opportunities  about 
educational  research,  methods  and  procedures.  And  faculty  members  need  to  continue  to 
deepen  their  understanding  of  learning  theory  and  teaching  methods. 

Regarding  the  classroom  research 

Each  faculty  engaged  in  formal  experimental  classroom  research.  Therefore,  this  program 
aspect  was  very  successful.  The  research  designs  resulted  from  the  faculty  members  working 
with  the  program  leaders  to  determine  their  interests  in  their  first  attempt  at  educational 
research  with  students  in  the  classroom.  Once  their  theoretical  interests  were  determined,  a 
design  was  generated  and  appropriate  methodology  and  procedures  were  determined. 
(Understand  that  this  was  the  primary  goal,  the  design  and  execution  of  formal  educational 
research  in  the  classroom.)  Although  the  results  of  the  research  were  viewed  as  very 
important,  that  was  truly  secondary  for  us,  as  we  wanted  to  bring  the  faculty  members  into 
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the  educational  research  arena  by  preparing  them  as  well  as  could  be  expected  on  theory  and 
best  practices  in  just  18  days.  Of  course,  we  understand  that  a deep  understanding  of  theory 
and  practice  cannot  result  from  18  days,  regardless  of  how  dedicated  the  learners  are. 
However,  they  learned  a lot  and  are  interested  in  learning  more  and  deepening  their 
understanding  about  the  theory.  Therefore,  the  goal  of  engaging  in  research  was  our  first 
priority  goal.  Professors  were  committed  to  their  classroom  research  and  were  serious  about 
their  research  focus.  However,  regarding  student  learning  for  each  individual  study,  there 
was  no  significant  learning  gain  across  the  studies.  Of  course,  that  is  not  “failure”  for  us, 
although  if  not  careful,  faculty  members  engaging  in  educational  research  for  the  first  time 
could  perceive  it  that  way.  Most  important  is  that  they  explain  their  results.  Thus,  each 
professor  produced  a research  manuscript  describing  his/her  research,  design,  methodology, 
procedures,  results  and  conclusions. 

What  is  important  to  discuss  are  the  following  variables 
Standard  End  of  Course  Evaluations 

Although  not  a realistic  expectation,  we  had  hoped  that  students  in  the  redeveloped  2006 
course  would  evaluate  the  course  more  positively  than  students  in  the  2005  courses.  In  using 
the  regular,  standard  course  evaluation  forms  (used  each  semester),  there  was  no  significant 
difference  between  the  two  evaluations.  Recall  that  the  professors  invited  to  participate 
varied  in  their  standard  teaching  evaluation  scores.  And  although  some  scores  shifted 
somewhat,  these  shifts  could  be  normal  shifts  and  were  not  significant  enough  to  reflect 
differences  that  could  be  attributed  to  the  treatment  variables.  This  research  aspect  needs  to 
be  considered  and  designed  into  a study  more  formally;  also  it  needs  to  be  tracked  across 
many  semesters  and  studied  longitudinally. 

New  End  of  Course  Questionnaires 

This  questionnaire  was  developed  for  several  purposes.  Its  content  focused  comprehensively 
on  the  content  within  the  faculty  development  program  components  (e.g.,  teaching  models 
and  styles,  student  learning  styles,  course  quality,  syllabus,  assessment,  and  more).  It  was 
developed  to  reflect  the  whole  program.  Therefore,  it  was  a “bank”  of  all  possible  questions 
one  could  ask  students  about  the  course  to  see  if  students  noted  changes  during  their  course 
experience  regarding  the  new  strategies,  models,  styles,  products,  behavior,  and  more  that 
professors  learned  about  in  the  faculty  development  program  and  may  be  trying  or 
improving.  The  questionnaire  was  administered  to  the  2005  class  as  a baseline  and  then  again 
to  the  2006  class,  different  students.  Professors  were  provided  a script  and  instructions  for 
administration  of  the  questionnaire.  We  acknowledge  that  because  of  the  comprehensiveness 
of  the  questionnaire,  it  was  too  long  to  be  used  generally,  but  we  had  to  begin  somewhere. 
However,  there  was  a significant  positive  change  between  the  2005  administration  of  the 
questionnaire  and  the  2006  administration  of  the  questionnaire. 


The  participating  professors  were  also  asked  to  complete  this  form  in  2005  and  2006  from 
the  perspective  of  how  they  felt  the  students  might  feel  about  their  courses,  instruction,  etc. 
The  same  individual  scanning  the  student  questionnaires  also  scanned  the  professors’ 
questionnaires  for  both  2005  and  2006.  He  made  a less  intense,  but  similar  comment  about 
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the  2006  faculty  administration  of  the  questionnaire,  but  not  about  the  2005  administration. 
That  led  us  to  consider  some  questions  about  the  use  of  the  questionnaires.  Therefore,  we  met 
with  the  professors  and  discussed  the  questionnaires. 

1.  Was  the  administration  of  the  questionnaire  to  students  any  different  in  2005  than 
it  was  in  2006? 

Response:  No  really  any  difference. 

2.  Do  you  think  the  students  in  those  classes  took  the  questionnaire  seriously? 
Response:  Yes,  we  think  they  did. 

3.  Did  the  students  have  less  time  in  2006  to  complete  the  questionnaire? 

Response:  Not  really. 

4.  Did  you  complete  both  the  2005  and  2006  questionnaire  equally  seriously? 
Response:  Yes 

5.  Do  you  feel  that  the  responses  of  the  student  in  2006  should  be  considered 
suspiciously? 

Response:  No,  we  feel  the  results  valuable. 

6.  Do  you  feel  the  results  are  valid  and  that  you  can  use  them? 

Response:  Yes. 

The  professors  did  not  really  agree  with  the  individual  who  scanned  the  forms.  Their 
perspective  is  acknowledged  as  different.  They  felt  inclined  to  use  the  results  and  that  the 
results  informed  and  benefited  them.  So  we  noted  this  in  the  data  report  section.  However,  in 
further  discussing  that  questionnaire,  it  is  a very  good  bank  of  questions.  However,  we  would 
suggest  that  as  the  professors’  research  becomes  focused  on  particular  aspects,  they  could 
choose  specific  questions  to  use  in  their  baseline  and  later  administration  of  the 
questionnaire.  This  would  also  result  in  a shorter  questionnaire  and  take  less  time  and 
concentration  for  students.  If  we  had  known  what  the  professors  were  going  to  focus  on 
during  the  research  semester,  we  could  have  done  that  with  the  2005  and  2006  classes. 
However,  since  the  professors  had  not  begun  the  faculty  development  program  at  the  time  of 
collecting  the  baseline  data  that  was  not  possible,  so  we  decided  to  administer  the  entire 
questionnaire,  fully  understanding  the  issue  of  length  and  time. 

Course  Grades 

We  did  not  ultimately  use  the  2005-2006  course  grades  as  a variable.  We  did  not  formally 
build  it  into  the  research  and  evaluation  with  specific  research  questions.  Therefore,  we  could 
attribute  any  course  grade  shifts  as  attributed  to  treatment  variables.  Finally,  we  and  the 
professors  were  interested  in  measuring  student  learning  for  the  entire  course,  but  we  did  not 
build  pre-  and  post-course  assessment  of  learning  into  their  studies.  We  have  suggested  that 
they  can  easily  add  that  measure  to  their  studies,  either  replications  or  new  studies,  and 
encouraged  them  to  do  so. 
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Conclusions 

The  faculty  development  program  was  successful  in  content  and  process.  The 
classroom  research  was  also  successful. 


Recommendations 

1.  Shorten  the  faculty  development  for  sustainability. 

2.  Use  the  student  questionnaire  as  a bank  of  questions  from  which  to 
choose,  selecting  items  that  match  the  research  focus 

3.  Add  some  program  components  (e.g.,  student  teaming,  classroom 
management  and  student  conflict  resolution  or  management) 

4.  Add  more  program  and  time  on  cooperative  learning  and  educational 
research 

5.  Continue  learning  opportunities  to  extend  knowledge  on  learning  theory 
and  instructional  practices 

6.  Continue  the  formal  LC 
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TODAY’S  STUDENTS 

Jule  Dee  Scarborough,  Ph.D. 


When  considering  student  demographics,  we  present  the  traditional  statistical  backgrounds, 
educational  interests,  gender,  ethnicity,  and  other  descriptors  of  today’s  students  (Brown, 
2003;  Kuh,  2003;  Lovett,  2003).  Additionally,  we  discuss  student  demographics  from  other 
perspectives  that  are  more  subjective.  Central  to  our  concerns  - Engineering  (and 
Technology),  we  include  the  U.S.  Department/National  Science  Foundation’s  (1998)  study 
on  Women  and  Men  of  the  Engineering  Path,  Eight  and  Cox’s  (2001)  findings  from  1600 
student  interviews  regarding  their  preferences  about  education,  and  Howe  and  Strauss’s 
(2000)  in-depth  discussion  of  the  Millennials.  Eevine  and  Cureton  (1998)  consider  the 
Generation  Xers  in  comparison  to  Eevine’ s (1980)  previous  study  of  the  Boomers.  These 
studies  focus  more  on  attitudes,  feelings,  insights,  life  experiences,  difficulties,  and  demands. 
To  us,  these  are  the  more  significant  demographics  if  we  are  to  know  our  students:  what  they 
are  about,  what  their  lives  are  demanding  of  them,  etc.  In  addition,  we  will  present  Brown’s 
(2000)  and  Tapscott’s  (1998)  perspectives  about  students  who  are  growing  up  digital,  as 
these  studies  or  perspectives  should  be  read  by  every  professor  and  every  administrator 
because  they  reveal  more  about  our  students’  real  capabilities  in  the  classroom  than  simple 
objective  statistical  demographic  descriptors.  (The  research  reported  here  also  deeply 
supports  the  teaching  and  learning  changes  targeted  by  our  initiative.) 

Although  we  have,  probably,  a higher  number  of  Millennials  entering  our  classrooms  today 
and  for  the  next  few  years,  there  are  also  many  Boomers  returning,  and  many  Generation 
Xers  either  returning  or  coming  to  college  after  working  for  some  years.  By  1994,  “44%  of 
all  college  students  were  over  twenty-five  years  old;  54%  were  working;  55%  were  female; 
and  43%  were  attending  part  time;  fewer  than  one  in  six  of  all  current  undergraduates  [at  that 
time]  fit  the  traditional  stereotype  of  the  American  college  student  attending  full  time,  being 
eighteen  to  twenty-two  years  of  age,  and  living  on  campus”  (U.S.  Department  of  Education, 
1996b  as  cited  in  Eevine  & Cureton,  1998,  p.  49).  Therefore,  all  three  generations  and  other 
contextual  factors  are  considered  and  woven  into  the  information  presented  below.  However, 
for  those  who  may  question  the  more  qualitative  approach  to  defining  student  profiles,  the 
tables  below  first  present  the  typical  statistical  information  on  our  CEET  students. 


Table  A.4.1:  Percentages  of  Transfer/Native  Undergraduates  in  Fall  2006 

College 

BUS 

EDU 

VPA 

LAS 

HHS 

EET 

All  Undergraduates 

Transfer  Students 

34.8 

46.2 

36.6 

44.0 

40.2 

38.1 

39.2 

Native  Students 

65.2 

53.8 

63.4 

56.0 

59.8 

61.9 

60.8 

Total 

100.0 

100.0 

100.0 

100.0 

100.0 

100.0 

100.0 

1 


Table  A.4.2:  CEET  Student  Demographics 


2005 
Major 

2006 
Major 

FEMALE 

MALE 

TOTAL 

2005-2006  ACT  Composite 

Mean 

SD 

N 

ELE 

21 

15 

295 

251 

316 

266 

23.02 

23.62 

3.70 

3.56 

223 

184 

7% 

6% 

93% 

94% 

100% 

100% 

ING 

11 

14 

71 

59 

82 

73 

23.61 

23.37 

3.48 

3.79 

56 

51 

13% 

19% 

87% 

81% 

100% 

100% 

MEE 

30 

23 

388 

393 

418 

416 

24.14 

24.27 

3.53 

3.51 

332 

314 

7% 

6% 

93% 

94% 

100% 

100% 

TECH 

20 

15 

317 

291 

337 

306 

21.00 

21.22 

3.72 

3.56 

230 

202 

6% 

5% 

94% 

95% 

100% 

100% 

UNDE- 

CH)ED 

E&ET 

2 

3 

17 

68 

19 

71 

22.38 

24.07 

4.51 

3.49 

16 

57 

11% 

4% 

89% 

96% 

100% 

100% 

Total 

84 

70 

1088 

1062 

1172 

1132 

22.94 

23.29 

3.85 

3.75 

857 

808 

7% 

6% 

93% 

94% 

100% 

100% 

2 


Table  A.4.2:  CEET  Student  Demographics  (continued) 


2005 
Major 

2006 
Major 

AMERICA 
N INDIAN 
OR 

ALASKAN 

NATIVE 

ASIAN  OR 
PACIFIC 
ISLANDER 

BLACK- 
NOT  HISPANIC 

HISPANIC 

NOT 

GIVEN 

OTHER 

WHITE/NOT- 

HISPANIC 

Total 

ELE 

1 

44 

31 

42 

36 

26 

14 

7 

3 

6 

4 

190 

178 

316 

266 

0% 

0% 

14% 

12% 

13% 

14% 

8% 

5% 

2% 

1% 

2% 

2% 

60% 

67% 

100% 

100% 

ING 

9 

10 

9 

7 

10 

7 

3 

2 

3 

49 

46 

82 

73 

0% 

0% 

11% 

14% 

11% 

10% 

12% 

10% 

4% 

0% 

2% 

4% 

60% 

63% 

100% 

100% 

MEE 

2 

44 

47 

23 

28 

28 

28 

7 

7 

6 

6 

310 

298 

418 

416 

0% 

0% 

11% 

11% 

6% 

7% 

7% 

7% 

2% 

2% 

1% 

1% 

74% 

72% 

100% 

100% 

TECH 

22 

18 

69 

59 

23 

22 

3 

3 

4 

5 

216 

199 

337 

306 

0% 

0% 

7% 

6% 

20% 

19% 

7% 

7% 

1% 

1% 

1% 

2% 

64% 

65% 

100% 

100% 

UNDE- 

CH)ED 

E&ET 

1 

3 

3 

8 

2 

5 

1 

2 

12 

53 

19 

71 

5% 

0% 

0% 

4% 

16% 

11% 

11% 

7% 

5% 

0% 

0% 

3% 

63% 

75% 

100% 

100% 

Total 

2 

2 

119 

109 

146 

138 

89 

76 

21 

13 

18 

20 

777 

774 

1172 

1132 

0% 

0% 

10% 

10% 

12% 

12% 

8% 

7% 

2% 

1% 

2% 

2% 

66% 

68% 

100% 

100% 

3 


Table  A. 4. 3:  Cumulative  GPA 


2005-2006  Cumulative  GPA 

Mean 

SD 

N 

ELE 

2.53 

2.52 

0.75 

0.75 

314 

263 

ING 

2.46 

2.50 

0.80 

0.86 

82 

73 

MEE 

2.59 

2.54 

0.76 

0.78 

414 

407 

TECH 

2.64 

2.68 

0.71 

0.75 

335 

304 

UNDE- 

CIDED 

E&ET 

2.46 

2.35 

0.75 

0.90 

19 

68 

Grand 

Total 

2.58 

2.56 

0.75 

0.78 

1164 

1115 

Table  AAA:  Semester  GPA 


2005-2006  Semester  GPA 

Mean 

SD 

N 

ELE 

2.36 

2.33 

1.01 

1.03 

314 

263 

ING 

2.42 

2.49 

1.05 

1.08 

82 

73 

MEE 

2.45 

2.34 

0.97 

1.01 

414 

407 

TECH 

2.73 

2.77 

1.00 

1.00 

335 

304 

UNDE- 

CIDED 

E&ET 

2.11 

2.29 

1.07 

1.00 

19 

68 

Grand 

Total 

2.50 

2.46 

1.01 

1.03 

1164 

1115 

Table  A.4.5:  HS  Pereentile  Rank 


2005  -2006  HS  Percentile  Rank 

Mean 

SD 

N 

ELE 

0.65 

63.82 

0.20 

20.23 

207 

165 

ING 

0.63 

63.31 

0.19 

18.79 

57 

48 

MEE 

0.68 

66.98 

0.18 

19.08 

305 

293 

TECH 

0.60 

59.04 

0.19 

19.61 

221 

199 

UNDE- 

CIDED 

E&ET 

0.54 

67.53 

0.23 

19.63 

17 

59 

Grand 

Total 

0.64 

64.04 

0.19 

19.72 

807 

764 
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Table  A. 4. 6:  Financial  Aid 


Financial  Aid  FY  1977-2006 

Grants,  Scholarships, 
Other  Gifts 

Loans 

Employ 

/ment 

Total  Financial  Aid 

% of  UG 

Fiscal 

Award 

per 

Award 

per 

Award 

per 

Award 

per 

Receiving 

Year 

Recipients 

Recipient 

Recipients 

Recipient 

Recipients 

Recipient 

Recipients 

Recipient 

Aid 

1985 

6,043 

1,420 

6,448 

2,086 

4,843 

696 

12,485 

2,035 

70.7 

1986 

6,077 

1,712 

6,560 

2,058 

6,207 

602 

11,766 

2,349 

64.6 

1987 

6,091 

1,696 

5,747 

1,879 

5,021 

731 

10,475 

2,368 

56.8 

1988 

5,925 

1,900 

5,426 

1,864 

4,574 

781 

9,854 

2,531 

52.0 

1989 

5,988 

2,019 

4,417 

2,107 

4,179 

858 

9,618 

2,597 

53.1 

1990 

6,358 

2,243 

4,497 

2,300 

4,220 

967 

10,753 

2,667 

59.6 

1991 

6,472 

2,332 

5,193 

2,429 

4,715 

920 

11,449 

2,799 

62.8 

1992 

6,102 

2,872 

5,083 

2,838 

4,235 

1,007 

9,879 

3,666 

54.2 

1993 

6,577 

3,148 

4,266 

3,266 

5,003 

841 

10,067 

3,858 

59.9 

1994 

6,683 

3,221 

5,812 

3,634 

4,303 

1,067 

10,432 

4,528 

62.1 

1995 

6,946 

3,337 

6,113 

3,919 

4,269 

1,061 

10,847 

4,763 

66.0 

1996 

6,955 

3,490 

6,579 

4,110 

4,076 

1,103 

10,876 

5,136 

70.7 

1997 

7,014 

3,795 

7,177 

4,029 

4,044 

1,186 

11,226 

5,374 

73.0 

1998 

9,008 

3,099 

7,303 

4,297 

4,364 

1,362 

11,101 

5,877 

70.0 

1999 

9,274 

3,350 

7,678 

4,630 

4,540 

1,316 

11,489 

6,318 

70.3 

2000 

9,432 

3,484 

8,035 

4,799 

4,258 

1,687 

11,764 

6,682 

69.6 

2001 

9,895 

3,808 

8,211 

5,203 

4,317 

1,651 

12,161 

7,197 

70.9 

2002 

10,059 

4,079 

8,529 

5,491 

4,030 

1,748 

12,484 

7,602 

71.5 

2003 

10,174 

4,088 

9,292 

5,699 

3,667 

1,827 

12,864 

7,871 

71.1 

2004 

10,537 

4,217 

9,851 

6,179 

3,763 

1,883 

13,238 

8,490 

72.5 

2005 

10,442 

4,186 

9,971 

6,882 

3,841 

2,053 

13,175 

9,124 

73.1 
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Figure  A.4.1 : Recipients  of  Undergraduate  Financial  Aid,  FY  1977-2005 


Students  Receiving  Financiai  Aid 


Figure  A. 4. 2:  Undergraduate  Enrollment  by  Racial/Ethnic  - Fall  1996  - Fall  2006 


Black  — ■ — Hispanic  Asian 
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NIU  Freshmen  Survey  Summary 


The  statements  below  provide  a more  personal  perspective  about  CEET  students.  Only  the 
summary  statements  that  mention  CEET  students  were  selected  from  the  NIU  report,  as  those 
reflect  the  self  perspectives  of  potential  engineering  and  technology  majors  who  responded  to 
the  NIU  Ereshman  Survey  (House  & Xiao,  2006,  pp.  5-9): 

1.  The  percentage  of  NIU  freshmen  who  rated  themselves  as  above  average  or  top  ten  percent  on  computer  skills 
was  highest  for  the  College  of  Engineering  and  Engineering  Technology  (57.5  %)  and  was  lowest  for  the 
College  of  Health  and  Human  Sciences  (33.3%). 

2.  The  percentage  of  NIU  freshmen  who  rated  themselves  as  above  average  or  top  ten  percent  on  drive  to  achieve 
was  highest  for  the  College  of  Health  and  Human  Sciences  (76.1%)  and  was  lowest  for  the  College  of 
Engineering  and  Engineering  Technology  (64.7%). 

3.  The  percentage  of  NIU  freshmen  who  rated  themselves  as  above  average  or  top  ten  percent  on  mathematical 
ability  was  highest  for  the  College  of  Engineering  and  Engineering  Technology  (76.5%)  and  was  lowest  for  the 
College  of  Education  (25.8%). 

4.  The  percentage  of  NIU  freshmen  who  rated  themselves  as  above  average  or  top  ten  percent  on  writing  ability 
was  highest  for  the  College  of  Liberal  Arts  and  Sciences  (59.4%)  and  was  lowest  for  the  College  of  Engineering 
and  Engineering  Technology  (37.3%). 

5.  The  percentage  of  NIU  freshmen  who  indicated  that  their  highest  degree  planned  at  NIU  was  a master’s  degree 
was  highest  for  the  College  of  Engineering  and  Engineering  Technology  (51.0%)  and  was  lowest  for  the 
College  of  Visual  and  Performing  Arts  (10.7%). 

6.  The  percentage  of  NIU  freshmen  who  indicated  that  a very  important  reason  for  selecting  NIU  was  a teacher’s 
advice  was  highest  for  the  College  of  Visual  and  Performing  Arts  (12.8%)  and  was  lowest  for  the  Colleges  of 
Engineering  and  Engineering  Technology  (2.7%). 

7.  The  percentage  of  students  who  rated  that  the  cost  of  attending  this  college  was  a very  important  reason  for 
selecting  NIU  was  highest  for  the  College  of  Health  and  Human  Sciences  (42.8%)  and  was  lowest  for  the 
College  of  Engineering  and  Engineering  Technology  (31.8%). 

8.  The  percentage  of  students  who  indicated  that  a very  important  reason  for  selecting  NIU  was  the  advice  of  a 
high  school  guidance  counselor  was  highest  for  the  College  of  Education  (14.2%)  and  was  lowest  for  the 
College  of  Engineering  and  Engineering  Technology  (2.7%). 

9.  The  percentage  of  students  who  indicated  that  size  of  college  was  a very  important  reason  for  selecting  NIU 
was  highest  for  the  College  of  Liberal  Arts  and  Sciences  (34.4%)  and  was  lowest  for  the  College  of  Engineering 
and  Engineering  Technology  (24.0%). 

10.  The  percentage  of  students  who  rated  their  chances  as  very  good  that  they  would  transfer  to  another  school  was 
highest  for  the  Engineering  and  Engineering  Technology  (9.3%)  and  was  lowest  for  the  College  of  Health  and 
Human  Sciences  (2.9%). 

11.  The  percentage  of  students  who  rated  their  chances  as  very  good  that  they  would  be  satisfied  with  their 
institution  was  highest  for  the  College  of  Education  (56.7%)  and  was  lowest  for  the  College  of  Engineering  and 
Engineering  Technology  (36.8%). 

12.  The  percentage  of  students  who  indicated  that  they  considered  it  very  important  or  essential  to  influence  the 
political  structure  was  highest  for  the  College  of  Liberal  Arts  and  Sciences  (28.5%)  and  was  lowest  for  the 
College  of  Engineering  and  Engineering  Technology  (15.1%). 

13.  The  percentage  of  students  who  indicated  that  they  considered  it  very  important  or  essential  to  influence  social 
values  was  highest  for  the  College  of  Liberal  Arts  and  Sciences  (50.2%)  and  was  lowest  for  the  College  of 
Engineering  and  Engineering  Technology  (29.3%). 
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14.  The  percentage  of  students  who  indicated  that  they  considered  it  very  important  or  essential  to  help  others  in 
difficulty  was  highest  for  the  College  of  Health  and  Human  Sciences  (81.1%)  and  was  lowest  for  the  College  of 
Engineering  and  Engineering  Technology  (53.3%). 

15.  The  percentage  of  students  who  indicated  that  they  considered  it  very  important  or  essential  to  develop  a 
meaningful  philosophy  of  life  was  highest  for  the  College  of  Visual  and  Petforming  Arts  (50.3%)  and  was 
lowest  for  the  College  of  Engineering  and  Engineering  Technology  (35.8%). 

16.  The  percentage  of  students  who  indicated  that  they  considered  it  very  important  or  essential  to  participate  in 
community  action  program  was  highest  for  the  College  of  Liberal  Arts  and  Sciences  (24.7%)  and  was  lowest  for 
the  College  of  Engineering  and  Engineering  Technology  (17.3%). 

17.  The  percentage  of  students  who  indicated  that  they  considered  it  very  important  or  essential  to  promote  racial 
understanding  was  highest  for  the  College  of  Visual  and  Performing  Arts  (42.9%)  and  was  lowest  for  the 
College  of  Engineering  and  Engineering  Technology  (20.7%). 


General  Characteristics 


Boomers 

After  studying  college  students  of  the  1970s  (the  Boomers),  Levine  (1980)  reported  the 
results  in  When  Dreams  and  Heroes  Died.  His  report  portrayed  them  as  optimistic  about  their 
own  futures,  but  not  about  the  country  - having  a rather  dooms-day  perspective.  Thus  they 
turned  inward,  focused  on  themselves  and  their  vocations,  making  material  goods  of  great 
importance.  Vietnam  and  Watergate  set  the  tone  of  their  perspectives,  slightly  or  greatly 
negative.  This  generation  was  short  on  heroes,  and  their  political  attitudes  were  conservative. 
They  were  less  well  prepared  academically;  the  social  sense  of  community  on  campuses 
declined;  individualism  dominated.  Individual  sports  grew;  students  became  more  socially 
liberal;  clubs  developed  around  gender,  race,  religion,  and  ethnical  differences.  Individual 
rights  and  freedoms  became  more  important,  and  liberalization  of  sex,  divorce,  marijuana, 
etc.  increased.  Thus  they  turned  inward,  focused  on  themselves  and  their  vocations,  making 
material  goods  of  great  importance.  Levine  (1980)  continued  to  study  college  students  during 
the  1980s,  asking  the  same  set  of  questions.  They  kept  giving  him  essentially  the  same 
answers  with  an  exception.  A higher  proportion  said  it  was  essential  or  very  important  to  be 
very  well  off  financially  and  the  number  who  felt  that  way  kept  rising  throughout  the  study. 

Gen  X 

When  continuing  with  the  1990s  students,  the  answers  began  to  show  “dramatic  changes 
regarding  optimism  about  the  future,  social  involvement,  and  life  goals”  (Levine  & Cureton, 
1998,  p.  xiii).  Levine  was  joined  by  a colleague,  Cureton,  and  together  they  repeated  the 
research  of  the  70s  study.  When  Hopes  and  Fear  Collide  (1998)  presents  their  findings. 
Beginning  with  the  issues  of  the  “Generation  without  a name”  (Generation  X,  the  Lost 
Generation,  or  the  13*  Generation),  these  students,  bom  in  the  late  70s  and  early  80s, 
attended  college  in  the  mid-  to  late- 1990s.  We  have  had  them  in  our  classrooms  for  a number 
of  years  now  (a  small  generation,  the  upbeat  and  downwardly  mobile  generation,  the  “we 
don’t  have  a clue  generation”)  and  were  described  as  optimistic  about  their  future,  having 
expectations  to  live  as  well  as  their  parents,  and  feeling  they  had  career  potential;  most 
importantly,  they  rejected  the  claims  of  the  ‘me’  generation.  They  were  not  optimistic  about 
the  country’s  future  and  were  focused  on  having  interesting  lives  where  personal  preferences 
took  first  place  over  a job.  They  believed  education  was  the  pathway  to  money  as  well  as 
personal  growth.  Close  approximations  of  this  description  were  published  both  by  Fortune 
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and  Time  magazines  (Homblower,  1997  as  cited  in  Levine  & Cureton,  1998,  p.  5;  see  also 
Deutschman,  1992);  however,  Business  Week  described  the  same  generation  as 

shut  out,  angry,  neglected,  and  pessimistic  about  their  personal  futures. . .destined  for 
mundane  and  marginally  challenging  work  that  provides  a paycheck  and  little 
else. . .more  likely  than  past  generations  to  be  unemployed,  underemployed,  and  living 
at  home  after  completing  school. . .a  generation  that  resented  the  baby  boomers  for 
blocking  their  career  paths. . .economically  at  risk. . .emotionally  unstable.  (Zinn, 

1992,  p.76  as  cited  in  Levine  & Cureton,  pp.  5-6). 

Levine  and  Cureton  felt  there  was  some  truth  in  both  perceptions. 

Brandweek  called  Gen  Xers  “a  complex  mix  of  contrasts”  (Benezra,  1995,  p.34  as  cited  in 
Levine  & Cureton,  1998,  p.  7).  As  this  group  is  somewhat  waning  in  our  classrooms,  I will 
spend  less  time  on  them  than  the  following  group,  the  Millennials;  however,  we  are  still 
teaching  the  Xers,  so  some  time  on  them  is  still  important.  This  generation  lived  through 
profound  change:  demographically,  socially,  globally,  economically,  and  technologically. 
They  are  the  smallest  generation;  the  racial  numbers  shifted,  with  Hispanics  growing  by  42% 
and  Asian  Americans  rising  by  100%  - a sharp  rise  in  the  number  of  teenagers  of  color.  This 
generation  has  lived  in  a racially  and  ethnically  politically  charged  world,  dramatically 
different  than  their  parents  or  the  adult  generations,  and  they  were  not  prepared  to  respond  to 
the  new  conditions.  They  endured  the  first  quarter  of  the  “American  Century,”  where  the 
U.S.  economic  supremacy  waned,  the  national  trade  debt  soared,  dollar  declined,  purchasing 
power  parity  shifted  and  declined,  trade  balances  shifted  negatively,  consumer  prices  soared, 
and  poverty  rates  increased,  with  those  in  poverty  increasing  about  50%.  They  were  in 
college  at  time  when  the  nation’s  economy  worsened.  This  caused  them  to  worry  greatly 
about  their  futures,  jobs,  economic  security  (their  and  the  country’s),  being  able  to  afford  a 
home  and  family,  and  technological  change  at  an  unprecedented  rate,  as  well  as  national 
failures  of  great  magnitude  (e.g.  chemical  leaks,  nuclear  accidents,  oil  spills,  a space  shuttle 
explosion,  politically  charged  international  events  with  other  countries,  and  environmental 
issues).  Also  turbulent  were  global  changes  such  as  unification,  atomization,  alliances,  civil 
collapses  - ethic  cleansing,  international  trading  bloc  development,  and  wars. 

Communication  changed  dramatically  - the  quiet  revolution.  This  technology  brought  the 
accidents,  failures,  and  catastrophes  around  the  world  to  them,  while  they  were  very  young, 
right  into  their  homes  more  dramatically  than  any  other  generation  had  experienced.  They 
came  to  fear  disasters  as  they  questioned  whether  they  would  be  able  to  live  in  such  a world. 
There  seemed  to  be  no  order  to  things  in  their  lives.  And,  in  fact,  a new  world  order  was 
formed  as  they  grew  into  adults,  where  the  position  of  the  US  could  no  longer  be  clearly 
defined  and  seemed  far  from  as  strong.  Their  “streets  became  more  dangerous”;  the  place  of 
religion  in  their  lives  diminished.  Everything  seemed  to  be  changing  for  the  worse.  Levine 
and  Cureton  compare  the  time  of  the  Generation  Xers  to  the  “Rip  Van  Winkle”  story  where 
he  slept  for  20  years  and  woke  up  to  a completely  changed  environment  - everything  was 
strange.  They,  however,  remain  energized  by  their  desire  for  a good  life  and  to  make  their 
world  a better  place.  “This  is  a generation  in  which  hope  and  fear  are  colliding”  (p.  17). 

Levine  and  Cureton  (1998)  went  on  to  study  undergraduate  students  between  1993  and  1997. 
When  asked  what  social  and  political  events  had  an  impact  on  their  lives,  students  responded 
consistently  with  negative  examples  and  expressed  the  situation  as 
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• Our  experience  is  of  flaws,  problems,  and  decline.  We’re  not  number  one  in  anything. 

Our  generation  grew  up  with  that. 

• The  world  seems  to  be  falling  apart. 

• We  don’t  have  anything  that  [is]  stable  to  hold  on  to. 

• Rome  and  Greece  fell;  so  can  the  U.S.  (p.  28) 

Their  greatest  and  strongest  criticism  was  of  the  government  and  political  system  - believing 
that  nothing  works  and  that  social  change  cannot  happen  with  the  present  politics.  Family 
values  are  breaking  down,  people  are  self  focused,  the  news  is  biased,  and  private 
corporations  are  more  concerned  about  profits  than  public  responsibility.  Doctors  are  too 
motivated  by  money,  not  enough  by  helping  people.  Political  leaders  do  not  have  integrity, 
and  executives  deserve  the  high  salaries  they  are  receiving.  Congress  has  the  people’s 
interests  at  heart.  However,  these  same  students  are  very  optimistic  about  the  future:  “I 
expect  things  to  get  worse  before  they  get  better,”  “I’m  pragmatically  optimistic,”  and,  the 
favored  response  by  the  authors,  “I  am  cynically  optimistic”  (p.  29).  These  Gen  Xers’ 
responses  differed  greatly  from  the  Boomers  and  also  from  the  Millenniums  (Howe  & 

Strauss,  2000)  to  be  discussed  below.  The  Boomers’  responses  were  more  negative  and  less 
hopeful.  The  Gen  Xers  have  a greater  sense  of  efficacy,  believing  that  they  can  make  a 
difference  individually.  When  asked  how  they  could  be  so  negative  about  the  social  and 
political  institutions  yet  so  positive  about  their  future  and  change,  they  responded: 

• Our  generation  is  getting  more  involved. 

• The  younger  generation  will  is  more  concerned  with  the  planet. 

• Our  generation  will  be  able  to  fix  the  problem. 

• Our  generation  will  do  things. 

• Our  generation  works  hard.  We  will  do  something. 

• We  are  the  future. . .Our  generation  cares  about  the  country  and  society. 

• Our  generation  will  make  a revolution,  (p.  30) 

The  authors  felt  that  student  attitudes  did  not  fit  within  traditional  labels  and  that 
undergraduates  were  both  more  liberal  and  more  conservative.  What  might  be  called  the 
political  center  shifted,  and  although  they  seemed  to  be  polarized  ideologically,  their 
opinions  were  very  similar  - they  desired  change.  This  group,  Xers,  did  not  believe  in  the 
government’s  ability  to  fix  things  and  felt  they  had  to  make  the  changes.  Their  visions  were 
small,  pragmatic,  and  manageable,  “I  can’t  do  anything  about  the  theft  of  nuclear-grade 
weapons  materials  in  Azerbaijan,  but  I can  clean  up  the  local  pond,  help  tutor  a troubled  kid, 
or  work  at  the  homeless  shelter”  (p.  36).  This  differed  from  the  responses  of  the  70s  and  80s. 
The  90s  groups  named  heroes  who  were  more  local  and  that  they  personally  knew,  rejecting 
the  “larger  than  life”  people  as  “inaccessible. . .up  on  a pedestal”;  as  compared  with  the  70s 
and  80s,  whose  heroes  were  named  as  no  one,  God,  entertainers,  athletes. 

A significant  majority  of  students  in  the  1993  phase  of  the  studies  had  at  least  one  friend  of 
another  race;  were  comfortable  expressing  controversial  viewpoints;  were  comfortable  with 
interracial  dating;  felt  a sense  of  community  when  on  campus;  and  did  not  feel  uncomfortable 
socializing  with  students  different  racially  from  themselves.  In  1997,  however,  a majority  of 
deans  at  four  year  colleges  described  things  differently: 
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the  climate  on  campus  was  politically  correct;  civility  has  declined  on  the  college 
campus;  students  of  different  racial  and  ethnic  groups  do  not  often  socialize  together; 
reports  of  sexual  harassment  have  increased;  and  students  feel  uncomfortable 
expressing  unpopular  or  controversial  opinions.  And  although  not  the  majority,  a 
significant  number  of  deans  felt  that  there  is  greater  racial  tension  and  more 
victimization  and  that  diversity  is  the  main  cause  of  conflict  between  students  on 
campuses,  (pp.  71-72) 

The  deans’  perspectives  were  diametrically  different  than  how  the  students  responded. 

Levine  and  Cureton  (1998)  identified  four  characteristics  that  exacerbated  causes  of 
multicultural  tension:  Preoccupation  with  Differences;  Mitosis  of  Student  Groups,  meaning 
that  differences  among  students  were  growing,  resulting  in  students  defining  the  proportion 
of  students  as  being  “like  me”  as  reduced,  and  changing  their  perspectives  as  the  number  of 
students  who  are  “different  from  me”  is  increasing  in  number  (p.  85).  The  undergraduates 
responding  in  the  1993  study  viewed  themselves  as  lonely,  with  few  people  like  themselves. 
The  third  characteristic  was  that  there  was  great  segregation  on  campuses  - voluntarily 
segregated  with  “locations”  on  campus  belonging  to  particular  groups  and  where  students 
group  by  like  kind.  This  was  greater  the  larger  the  institution.  Finally,  the  fourth 
characteristic  was  the  growing  sense  of  victimization. 

A rising  percentage  of  undergraduates  felt  they  were  being  disadvantaged  to  the 
perceived  advantage  of  other  students.  For  example,  affluent  students  complained 
that  they  were  being  made  to  pay  higher  tuition  to  support  less  affluent 
undergraduates.  Poorer  students  said  admissions  standards  were  less  rigorous  for 
students  who  can  pay  full  tuition.  Men  pointed  fingers  at  women,  who  they  felt  had 
preference  in  entering  traditionally  male  professions,  and  women  pointed  back  at 
men,  talking  of  the  glass  ceiling  they  experienced.  Racial  majorities  charged  that 
minorities  were  being  advantaged  at  their  expense,  and  minorities  made  the  reverse 
claim.  International  students  were  critical  of  domestic  populations  and  vice  versa. 

One  religious  group  complained  about  the  preferential  treatment  given  the  next.  (pp. 
90-91) 

In  summary,  undergraduate  students  felt  they  were  being  treated  unfairly  and  others  were 
being  treated  better  at  their  expense,  whoever  they  were.  Focus  was  reported  on  differences 
rather  than  commonalities,  isolation  between  groups  was  unbridgeable,  with  greater 
separation  of  groups,  leading  possibly  to  “Hobbesian  worlds”  - all  against  each  other  (p.  91). 
Multiculturalism  was  the  greatest  issue  on  campuses  in  the  1990s  (and  possibly  still  today). 

In  my  more  diverse  classrooms,  it  is  extremely  difficult  to  get  particular  groupings  of 
students  to  break  apart  and  mix  more  in  the  required  group  activities.  I have  to  systematically 
struggle,  artificially,  to  make  that  happen  to  diversify  the  individuals  in  groups  for  group 
work. 

An  important  understanding  of  the  Generation  X (1990s)  students  is  reflected  in  their 
personal  lives.  For  the  Generation  Xers,  the  best  descriptor  over  all  is  “overwhelmed  and 
more  damaged”  (p.  91).  Classroom  impacts  range  from  higher  disruption  rates,  drug  abuse, 
alcohol  abuse,  and  gambling  and  suicide.  Deans  report  that  students  have  more  baggage, 
there  are  more  developmentally  disabled  students,  and  they  are  spending  more  time  on 
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emotionally-ill  students.  Students  have  more  nonacademic  issues,  student  discipline  takes 
more  time,  and  students  really  expect  the  community  to  help  make  things  right  for  them: 

• “I  don’t  have  a social  life.” 

• “There  is  no  free  time.” 

• “My  whole  life  is  juggling.” 

• “Study,  that’s  all  we  ever  do.” 

• “I’m  always  behind.  I never  catch  up.” 

• “I  feel  run  down.” 

• “People’s  lives  are  dictated  by  their  jobs.” 

• “The  high  cost  means  I have  to  work  forty  to  fifty  hours  per  week,  and  it’s  killing  me. 

Sometimes  I fall  asleep  in  class. . .life  is  just  work,  school,  and  home”  (pp.  99-100) 

Gen  Xers,  as  reported  in  the  1990s,  were  optimistic  about  their  futures  and  wanted  good  jobs, 
successful  relationships,  children,  and  to  live  as  well  as  their  parents  (the  Boomers).  Their 
fears  were  summed  up  as  nothing  was  tangible  for  them  to  hold  on  to.  They  were  more 
positive  than  their  predecessors,  and  somewhat  optimistic,  yet  they  were  worried  about  the 
government  and  politicians,  etc.  They  were  also  worried  about  their  private  lives  but  believed 
they  would  be  successful.  “This  is  a generation  that  is  desperately  clinging  to  its  dreams,  but 
their  hope,  though  broadly  professed,  is  fragile  and  gossamer-like.  Their  lives  are  being 
challenged  at  every  turn:  in  their  families,  their  communities,  their  nation,  and  their  world. 
What  is  remarkable  is  that  their  hopes  have  not  been  engulfed  by  their  fears”  (Levine  & 
Cureton,  1998,  p.  145).  In  concluding,  these  authors  label  this  generation  as  a transitional 
generation  in  a time  of  discontinuity,  a rare  time  of  profound  change  in  our  society  (and  the 
world).  Levine  and  Cureton  note  the  enormous  size  of  the  population  attending  college, 
which  reduces  the  benefit  of  a college  education,  ending  that  undergraduates  need  an 
education  that  provides  four  things:  hope,  responsibility,  an  appreciation  of  differences,  and 
efficacy  or  the  sense  that  one  can  make  a difference.  Levine  and  Cureton  end  with 
recommendations  for  a curriculum  that  includes  communication  and  thinking  skills,  with  a 
focus  on  fluency  in  words  and  numbers;  transitional  and  critical  thinking  skills;  and 
continuous  learning  with  creativity  as  highly  essential.  Students  must  learn  how  to  access 
knowledge  and  use  it,  knowledge  and  context  that  are  forever  changing  and  more  complex. 
The  other  elements  important  to  study  are  human  heritage,  past  and  present;  the  environment, 
as  this  is  a “green”  generation;  scientific  literacy;  an  understanding  of  the  multifaceted  roles 
that  we  play  as  individuals;  and  values  (because  if  students  are  to  value  differences  and 
respect  one  another  or  understand  why  cheating  is  wrong,  values  must  be  included  in  the 
curriculum).  Levine  and  Cureton’s  research,  although  primarily  about  Generation  X,  is  truly 
insightful  and  reveals  a great  deal  to  consider,  even  when  faculty  have  begun  to  teach  an 
entirely  new  generation,  the  Millennial  - those  who  have  grown  up  digitally. 

Millennials 

Howe  and  Strauss  (2000)  discuss  several  generations  in  several  different  books: 

Generations,  Generation  and  Millennials  Rising-The  Next  Great  Generation.  The  newest 
generation,  bom  from  1982  forward  and  now  accepted  as  the  “Millennials,”  is  in  college. 
Eighteen  in  2000,  they  either  have  just  or  are  about  to  graduate  from  college,  depending  upon 
the  number  of  years  in  college  (4-6).  From  a slightly  different  perspective,  Howe  and  Strauss 
argue  that  the  Millennials  are  quite  different  in  persona  than  the  Boomers  and  Xers.  Where 
the  two  previous  generations  tended  towards  “more  selfishness  in  personal  manner,  more 
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splintering  in  public  purpose,  more  profanity  in  culture  and  daily  discourse,  more  risk-taking 
with  sex  and  drugs,  more  apathy  about  politics,  and  more  crime,  violence,  and  social  decay, 
the  Millennials  - the  ‘good  news  revolution,’”  from  Howe  and  Strauss’  perspective,  represent 
a counter  culture  considered  to  be  the  “found”  generation  versus  the  “lost”  generation.  The 
Millennials  are  “optimists,”  more  upbeat,  confident,  and  positive,  and  the  number  who  worry 
about  violence,  sex,  and  drugs  is  rapidly  decreasing.  They  feel  that  growing  up  was  easier 
than  it  was  for  their  parents.  The  Millennials  are  cooperative  and  team  players  and  gravitate 
toward  group  activity  and  criticize  “selfishness.”  They  are  not  “self-absorbed”  and  accept 
authority.  They  identify  with  their  parents’  values  and  “trust  and  feel  close  to”  their  parents. 
Half  trust  the  government  to  do  what  is  right  most  of  the  time  and  believe  that  lack  of 
parental  discipline  is  a major  social  problem.  A large  majority  favor  “tougher  rules  against 
misbehavior  in  the  classroom  and  society  at  large”  (p.  8).  Millennials  are  “rule  followers,”  as 
the  homicide  rates,  violent  crime,  abortion,  and  pregnancy  among  teens  have  all  decreased 
very  significantly  in  the  last  five  years.  This  is  not  a “neglected”  generation,  as  “they  are  the 
most  watched  over  generation  in  memory”  (p.  9).  These  kids  have  more  structure  and 
supervision  by  parents  and  relatives,  teachers,  sitters,  etc.  as  well  as  more  curfews  and 
surveillance  cams  than  any  group  so  far  (pp.  8-9).  Millennials  are  also  smarter  than  we  think. 
Aptitude  test  scores  have  risen  with  all  racial  and  ethnic  groups,  and  this  generation  thinks  it 
is  cool  to  be  smart,  with  a record  number  taking  AP  tests,  and  “look[ing]  forward  to  going  to 
school,”  planning  to  attend  college.  There  is  an  increase  in  math  and  science  exam  scores, 
with  fourth  graders  near  the  top  internationally.  With  the  amount  of  homework  on  the 
increase  and  the  new  standards,  they  could  stay  on  top. 

The  Millennial  Generation  “believe  in  the  future  and  see  themselves  as  its  cutting  edge”  (p. 
10).  Why  are  they  so  different  than  expected?  Howe  and  Strauss  (2000)  feel  that  it  is  because 
the  predictive  assumption  that  the  future  will  be  a “straight-line  extension  of  the  recent  past 
that  never  happens”  is  wrong  (p.  10  ).  So  the  Millennial  Generation  is  just  as  the  other 
generations:  the  “Silent  Generation”  (the  Boomers’  parents),  the  Baby  Boomer  Generation, 
the  Generation  Xers  - very  different  from  the  others.  The  Millennials  rejected  the  name 
“Generation  Y”  or  “Echo  Boomers”  because  the  Generation  Y name  had  a negative 
connotation  and  did  not  express  their  uniqueness.  They  wanted  a “founding  word. . .that 
respects  their  newness,  a word  that  resets  the  clock  of  secular  history  around  their  own 
timetable. . .Millennial  acknowledges  their  technological  superiority  without  defining  them 
too  explicitly  in  those  terms. . .hints  at  what  their  rising  generation  could  grow  up  to 
become.... ‘We’re  the  kids  who  are  going  to  change  things’”  (pp.  10-12).  In  society,  the 
Millennials  have  been  regarded  as  special  since  birth  and  have  been  more  obsessed  over  at 
every  age  than  the  Xers,  which  partly  explains  why  harms  against  children  are  less  tolerated 
today  than  10  years  ago.  Political  issues  have  been  recast:  “kinder  politics,  as  in:  If  it’s  good 
for  children,  do  it  - and  if  it  isn’t,  don’t”  (p.  13). 

Howe  and  Strauss  (2000)  describe  this  generation  as  “numerous”  because  of  resurgent 
fertility  rates  and  large  families  from  a record  immigration  surge,  76  million  at  2000.  With 
2002  as  their  final  birth  year,  they  will  outnumber  Xers  as  well.  Add  that  to  additional 
immigration  numbers  and  they  may  become  America’s  first  100  million  person  generation. 
This  IS  America’s  most  ethnically  diverse  generation  and  least-Caucasian  generation.  One 
Millennial  in  every  five  has  at  least  one  non-citizen  parent;  this  is  possibly  the  largest 
second- generation  immigrant  group  in  U.S.  history,  and  they  are  the  first  group  to  think  of 
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itself  as  global  (p.  16).  America  is  “browning,”  irreversibly,  and  they  growing  up  thinking  of 
themselves  as  global.  This  is  and  can  be  an  amazing  group,  and  WE  are  responsible  for  their 
education,  development,  vision,  potential  and  ultimately  their  future.  Howe  and  Strauss 
discuss  in  great  depth  these  generations,  giving  in-depth  consideration  to  the  potential  and 
power  of  the  Millennials  as 

• Special.  . . .older  generations  have  inculcated  in  Millennials  the  sense  that  they  are, 
collectively,  vital  to  the  nation  and  to  their  parents’  sense  of  purpose. 

• Sheltered.  . . .are  the  focus  of  the  most  sweeping  youth  safety  movement  in  American 
history. 

• Confident.  With  high  levels  of  trust  and  optimism  - and  a newly  felt  connection  to 
parents  and  future  - Millennial  teens  are  beginning  to  equate  good  news  for 
themselves  with  good  news  for  their  country.  They  often  boast  about  their 
generation’s  power  and  potential. 

• Team-oriented.  From  Barney  and  soccer  to  school  uniforms  and  a new  classroom 
emphasis  on  group  learning,  Millennials  are  developing  strong  team  instincts  and 
tight  peer  bonds. 

• Achieving.  With  accountability  and  higher  school  standards  rising  to  the  very  top  of 
America’s  political  agenda,  Millennials  are  on  track  to  become  the  best  educated  and 
best  behaved  adults  in  the  nation’s  history. 

• Pressured.  Pushed  to  study  hard,  avoid  personal  risks,  and  take  full  advantage  of  the 
collective  opportunities  adults  are  offering  them,  Millennials  feel  a “trophy  kid” 
pressure  to  excel. 

• Conventional.  Taking  pride  in  their  improving  behavior  and  more  comfortable  with 
their  parents’  values  than  any  other  generation  in  living  memory,  Millennials  support 
convention  - the  idea  that  social  rules  can  help.  (pp.  43-44) 

The  trends  of  the  Millennials  will  become  more  obvious  with  each  birth  cohort  and  represent 
“a  direct  reversal  of  the  Boomers  and  Xers”  (Howe  & Strauss,  2000,  p.  84).  Names  clearly 
reflect  globalism  and  diversity.  Important  to  note,  but  rarely  discussed  regarding  immigration 
numbers,  is  that  there  are  also  significantly  more  Americans  being  born  abroad.  In  2000,  only 
2.4  million  Millennials  (3.5  percent  of  the  entire  generation)  were  themselves  immigrants. 
However,  approximately  14  million  Millennials  were  the  “children”  of  immigrants,  mostly 
Gen  Xers,  growing  by  half  since  1990  and  now  about  20  percent  of  this  generation  (it  was  6- 
8 percent  for  Boomers).  Therefore,  more  Millennials  can  be  expected  to  immigrate  to  the 
U.S.  Although  Gen  Xers  will  remain  the  largest  “first”  generation  immigrant  cohort  group  to 
preserve  their  native  cultures,  Millennials  are  going  to  be  a greater  “second”  generation 
immigrant  group  and  will  probably  be  “assimilationists”  (p.  84).  A large  number  of 
Millennials  will  be  the  children  of  foreign  born  Gen  Xers,  and  although  these  students  may 
have  to  endure  hardships  even  though  here  in  the  U.S.  (e.g.,  poverty,  no  health  care,  material 
hardships),  their  families  are  stable  and  have  a dedicated  work  ethic.  They  have  aspirations, 
will  do  well  in  school,  and  will  be  arrested  less  than  the  third  or  fourth  generation  of  the  same 
ethnic  group. 

Millennials  fit  the  general  pattern  of  second- generation  immigrants;  they  trade  the  language 
and  culture  of  their  parents’  nationalities  for  that  of  the  U.S.  Many  do  not  speak  English  at 
home,  but  they  show  more  “rapid  linguistic  assimilation”  across  all  ethnicities  and 
socioeconomic  levels  than  past  immigrant  children.  Because  Eatino  immigrants  have  become 
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so  large  in  number  and  have  such  a high  fertility  rate  (double  that  of  American  mothers  - two 
per  mother),  they  have  significantly  increased  the  Millennial  generation  birth  boom, 
especially  that  of  the  90s,  resulting  in  “a  geographic  nexus  of  Millennial  brownness”  (Howe 
& Strauss,  2000,  p.  85).  These  families  demographically  are  “mixed  status.”  They  have 
family  anchors  in  the  U.S.  because  of  having  a child  that  is  a U.S.  citizen,  but  the  parents 
may  not  be  citizens.  As  a result  of  the  changes  in  law  during  the  1990s,  the  “mixed  status” 
children  who  are  citizens  were  granted  an  advantageous  and  vast  array  of  cash  payments  and 
government  services  denied  to  their  undocumented  parents.  This  makes  Millennials  in  the 
U.S.  historically  by  far  the  most  racially  and  ethnically  diverse  generation;  nearly  35%  of  the 
Millennials  are  nonwhite  or  Latino  versus  14%  of  the  Boomers  (p.  85).  Not  much  of  the 
“browning”  or  change  comes  from  African  Americans,  whose  fertility  rate  has  fallen  during 
the  past  decades;  they  gain  little  from  the  immigration  influx.  The  “browning”  is  primarily 
Latino  with  some  Asian  influence.  As  a result,  African  Americans  are  no  longer  the  largest 
minority;  Latinos  have  replaced  them.  An  aside,  where  the  civil  rights  slogan  for  African 
Americans  has  been  “We  shall  overcome,”  the  civil  rights  slogan  quoted  from  the  Latina 
magazine  is  “Ours  is  going  to  be  ‘We  shall  overwhelm.’”  Regardless  of  the  race  or  ethnicity, 
Millennials  across  the  world  share  being  the  “more  wanted,  protected,  and  cared  for 
generation”  (p.  86). 

As  a result  of  growing  up  in  a politically  correct  environment,  Millennials  are  removed  from 
understanding  government,  causes,  etc.  What  they  have  heard  about  major  national/global 
civic  events  has  been  through  cynicism,  irony,  satire,  and  parody  - pop  culture.  This  made  it 
seem  that  not  much  is  really  at  stake.  That,  however,  does  not  mean  that  they  are  not 
respectful;  perhaps  they  cannot  relate  to  the  underlying  issues.  From  their  perspective,  the 
Millennials  feel  adult  generations  have  accomplished  some  good  that  has  enhanced  their 
(Millennials’)  lives  and  potential  futures.  Millennials  also  find  fault  with  the  self- 
centeredness  of  the  adult  generations,  often  feeling  that  the  nation  has  been  left  with  pointless 
arguments,  hypocrisy,  and  fragmentation  - pieces  that  do  not  fit  together  any  more.  This  is 
partly  why  the  Millennials  are  finding  a role;  they  have  civic  spirit  and  are  accomplishing 
things  crossing  racial  and  gender  boundaries,  engaging  in  community  service,  politics,  and 
faith.  They  have  already  begun  to  face  a different  set  of  challenges  than  that  of  the  adult 
generations. 

From  our  perspective  in  higher  education,  the  Millennials  may  represent  a somewhat  better 
educated  student,  one  better  prepared  for  college.  The  oldest  of  these  students  were  only  one 
year  old  when  the  report  Nation  at  Risk  came  out,  followed  by  the  national  calls  for  reform, 
e.g.  Goals  2000.  Hopefully,  as  reported  by  U.S.  News,  we  are  now  in  an  “age  of 
accountability,”  where  education  is  showing  some  improvement  and  a very  significant 
change  in  teacher  attitudes  (teachers  say  they  would  be  a teacher  again).  More  teachers  now 
have  master’s  degrees,  from  one  sixth  to  over  half  now.  Also  teaching  is  now,  again, 
considered  a profession  with  “meaning.”  Many  Teach  for  America  Gen  Xers  as  well  as  many 
bumed-out  Boomers  have  entered  the  classroom  to  make  a difference  with  their  lives  (Howe 
& Strauss,  2000,  pp.  146-147). 

We  are  now  teaching  students  who  are  and  have  been  digital  learners.  Brown’s  (2000)  article 
in  Change,  “Growing  up  Digital-How  the  Web  Changes  Work,  Education  and  the  Ways 
People  Learn,”  validated  some  prior  learning  theory  on  constructivism,  active  learning,  and 
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engagement  and  focused  on  the  World  Wide  Web  as  a transformative  medium  and  on  what 
that  means  for  learning.  The  students  are  linked  to  the  Internet  in  many  ways.  Tapscott 
(1998),  in  Growing  Up  Digital  - The  Rise  of  the  Net  Generation,  terms  the  digital  generation 
as  the  “N-Geners,”  today’s  media-literate  kids,  which  goes  far  beyond  technological  literacy 
in  using  computers  and  the  past  “uni-directional  technology”  that  was  often  “dumbed  down 
to  the  lowest  common  denominator”  (p.  2).  Digital-savvy  N-Geners  require  interactive 
media,  where  users  are  not  just  viewers  or  listeners  and  where  the  digital  wave  is  swallowing 
TV.  Children  today  are  watching  less  TV  as  new  media  penetrates  households.  Although 
Tapscott  seems  to  consider  Gen  Y as  the  digital  generation,  the  ages  he  discusses  seems  to 
include  some  Gen  Xers.  (Perhaps,  this  “Net  generation”  merges  those  from  the  X and 
Millennial  generations.)  However,  he  does  comment  that  Net  as  a demographic  codifies  and 
unifies  the  power  of  demographics  with  the  power  of  new  media  analysis,  breaking  from  the 
cynicism  surrounding  Gen  X.  He  feels  the  N Generation  defines  positively.  Boomers  were 
the  TV  generation,  which  at  the  time  reshaped  and  transformed  businesses,  communication, 
education,  and  more.  The  Net  has  now  caused  yet  another  profound  transformation  because 
of  its  wireless  capabilities  and  has  had  even  greater  impact  because  of  the  capability  for 
immediate  global  interaction  and  access  to  information.  This  greatly  changes  the 
demographics  of  the  students  in  our  classrooms.  The  Net  Generation  is  “breaking  free  from 
the  one-way,  centralized  media  of  the  past  and  is  beginning  to  shape  its  own  destiny.  Some 
argue  that  there  is  mounting  evidence  that  the  world  will  be  a better  place.  Technology  is 
transparent  to  them,  simply  the  vehicle  for  the  trip;  it  is  like  “air”  (p.  39).  However,  they  do 
want  to  know  how  computers  and  software  work  so  they  can  change  or  modify  it,  not  just  use 
it.  N-Geners  do  not  really  see  the  technology;  it  is  almost  irrelevant;  they  merely  see  all  the 
connections  they  can  make,  people,  information,  games,  applications,  services,  friends,  and 
protagonists  at  the  other  end  (p.  39).  They  do  not  register  the  “screen”;  they  see  what  is 
happening  and  the  people  and  information  that  the  screen  helps  them  to  access.  They  are 
more  focused  on  what  they  are  doing.  Digital  media  or  revolution  is  not  controlled  by  adults. 
Children  and  young  adults  are  becoming  experts  on  topics,  acquiring  knowledge  and 
experience  completely  differently,  and  they  are  being  taken  more  seriously  when  they  use 
what  they  have  learned.  They  can  engage  in  justified  argument  backed  up  with  the  latest 
information.  The  culture  has  also  been  transformed  because  of  the  Net  interaction. 

Descriptors  of  the  N-Gen  Culture  offered  by  Tapscott  are 

• Fierce  independence 

• Emotional  and  intellectual  openness 

• Inclusion 

• Free  expression  and  strong  views 

• Innovation 

• Preoccupation  with  maturity 

• Investigation 

• Immediacy 

• Sensitivity  to  corporate  interest 

• Authentication  and  trust 

They  can  challenge  corporations  on  value  and  results.  Also  because  anyone  can  say  anything 
(true,  accurate,  or  not),  they  are  seekers  of  authenticity  of  what  they  learn  from  the  Net. 
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Authenticity  leads  them  to  trust.  N-Gen  personalities  are  presented  as  accepting  of  diversity, 
a curious  generation,  and  assertive  and  self-reliant. 

They  are  a “contrarian”  generation  because  they  have  the  tools  to  question,  challenge,  and 
disagree;  they  are  critical  thinkers.  Tapscott  (1998)  feels  this  is  vital  to  the  future  of  humanity 
(p.  88).  The  Net  provides  the  venue  for  millions  to  argue  worldwide  and  in  real-time.  The  N- 
Geners  can  build  higher  self  esteem  with  Net  “friends.”  Although  not  always  positive,  they 
can  also  present  multiple  selves,  experiment  with  morphing  their  own  identities.  Tapscott 
feels  the  Net  builds  and  allows  the  use  of  multiple  intelligences  and  presents  the  results  of 
some  studies  that  show  if  students  can  have  access  to  computers  that  “alone  could  displace 
other  factors,  such  as  household  income,  in  improving  writing  skills”  (p.  99).  In  1998,  47% 
of  the  U.S.  population  demonstrated  low  or  no  levels  of  literacy.  Today’s  rates  are  reported  in 
four  categories:  below  basic  - illiterate  (14%),  basic  - can  understand  simple  documents  and 
brochures  (28%),  intermediate  - understand  articles  and  find  reference  materials  (43%), 
proficient  - capable  of  critical  thinking  (13%).  According  to  the  2002-2003  National 
Assessment  of  Adult  Literacy  - a division  of  the  National  Center  for  Education  Statistics 
(2006),  results  about  42%  of  adult  Americans  do  not  meet  the  full  prose  literacy  expectations 
of  high  school  graduates  and  that  rating  dropped  by  1%  by  1998  and  2003.  N-Geners  do 
seem  to  lack  appreciation  for  global  distances,  possibly  the  first  generation  to  have  a global 
perspective.  Spatial  orientation  with  the  immediate  environment  is  a different  question.  There 
is  a difference  in  thinking  and  processing  of  information  because  the  N-Gen  processes 
multiple  types  of  information  concurrently  and  less  sequentially;  they  can  organize  complex 
structures  of  information  and  also  link  to  other  sources  of  information  concurrently,  possibly 
freeing  them  from  linear  thinking.  The  link  between  input  and  output,  processing  - cognition 
and  reasoning  is  also  being  transformed.  This  can  result  in  more  easily  establishing 
relationships  between  knowledge  pieces  and  building  more  complex  schemas  where 
knowledge  is  more  easily  connected,  expanded,  and  used  across  more  widely  varying 
contexts. 

The  N-Geners,  however,  may  be  less  socially  skilled;  their  attention  spans  may  be  reduced, 
and  they  can  become  vain  about  their  prowess  (e.g.,  own  web  pages).  They  either  relieve 
stress  or  build  more  stress  by  interacting  with  the  Net,  as  its  use  consumes  time  in  an  already 
over-scheduled  day.  This  generation  has  become  addicted  to  the  Net.  Learning  is  changing, 
especially  learning  in  a knowledge-based  economy.  The  Net  has  challenged  education, 
content  and  process,  and  even  nine  years  after  Tapscott’ s (1998)  work,  we  are  only  just 
beginning  to  get  to  the  more  relevant  questions.  It  has  already  transformed  our  students’ 
learning,  and  although  we  are  making  progress,  our  content  and  process  has  not  been 
transformed  so  easily,  quickly,  or  well.  We  are  still  in  crises  in  K-12,  and  those  crises  have 
followed  into  higher  education  as  well.  The  entire  purpose  of  educational  institutions  is  at 
question.  At  the  heart  of  the  broader  question  is  that  learning  is  social  and  interactive.  The 
old,  yet  renewed,  best  practices  in  teaching  and  learning  are  active,  engaged,  interactive,  and 
social,  and  the  Net  has  only  reinforced  it  through  a different  media.  Interactive  learning  is  not 
linear;  it  has  to  engage  students  in  the  construction  of  knowledge  through  discovery  and 
moves  from  teacher-centered  to  learner-centered,  from  absorbing  material  to  “navigating” 
through  material  and  learning  how  to  learn.  It  is  built  throughout  one’s  lifetime,  not  just 
while  in  any  level  of  schooling.  Learning  has  always  been  about  customizing  rather  than  the 
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“one-size-fits-all”  methodology  we  have  been  using,  as  well  as  learning  as  fun  (not  torture) 
and  where  the  teaeher/professor  is  not  the  transmitter  but  a faeilitator. 

This  leads  us  back  to  Brown’s  article  (2000)  where  he  discusses  learning.  He,  along  with 
Tapscott  (1998),  goes  beyond  a discussion  of  learning  and  education  to  also  discuss  family, 
work,  economy,  and  play;  however,  we  will  only  consider  the  comments  more  directly 
related  to  our  initiative.  Brown  considers  the  Net  the  new  medium;  few  would  argue  that. 
But  he  goes  further  describe  its  process  as  being  more  two-way  than  books,  where  we  can 
“push  and  pull.”  The  one-way  approach  (books,  TV,  etc.)  pushes  information  at  us;  whereas, 
in  using  the  Net,  we  can  pull  what  we  want  from  it  as  well  and  can  take  learning  in  various 
directions  and  all  at  one  time.  The  second  aspect,  to  continue  one  of  Tapscott’ s,  is  that  the 
Web  honors  multiple  intelligences.  Literacy  has  typically  been  viewed  as  “text  focused.”  On 
the  web,  we  can  engage  in  the  abstract,  textual,  visual,  musical,  social,  and  kinesthetic, 
allowing  someone  to  engage  in  his/her  ideal  way  of  learning  and  also  provide  opportunities 
to  build  strengths  in  other  ways  of  learning.  Both  Brown  and  Tapscott  feel  that  the  Net 
realizes  the  opportunity  to  build  multiple  intelligences,  as  Gardner  (1983)  describes  them.  A 
third  aspect  of  the  Web  is  that  it  “leverages  the  small  efforts  of  the  many  with  the  large 
efforts  of  the  few”  (p.  12).  They  can  all  connect  on  the  Web  and  have  greater  impact.  Brown 
mentions  that  we  are  at  the  beginning  of  this  transformation,  and  even  five  years  later  that  is 
still  probably  true,  meaning  that  our  classrooms  and  the  student  demographics  will  be 
transformed  even  more.  Brown  and  Tapscott  both  feel  that  the  Web  or  Net  is  as  fundamental 
to  society  as  electrification.  Brown  feels  that  our  challenge  is  to  foster  an  “entrepreneurial 
spirit  toward  creating  learning  environments — a spirit  that  will  use  the  unique  capabilities  of 
the  Web  to  leverage  the  natural  ways  that  humans  learn”  (p.  13).  So  what  does  this  all  mean 
for  digital  learners?  Brown  led  a project  where  he  hired  15  year  olds  as  researchers,  giving 
them  two  jobs  - to  design  the  “workscape”  of  the  future  that  they  would  want  to  work  in  as 
well  as  the  school  or  “learningscape”  of  the  future.  The  designs  resulting  from  the  15  year 
olds  in  2000  “shook  them  up”  (p.  14).  They  were  “multiprocessors,”  doing  several  things  at 
once.  There  are  certainly  adults,  and  Boomer  adults,  who  also  fit  this  mode  - able  to  multi- 
process, but  probably  fewer.  Many  adults  are  linear  processors,  learners,  and  thinkers.  But, 
this  “parallel  processing”  that  we  speak  of  for  computers  is  equally  happening  with  those 
who  are  digitally  literate  and  who  are  interactive  on  the  Web.  Brown  found  that  the  15  year 
olds  operated  with  an  attention  span  between  30  seconds  and  5 minutes,  “paralleling  that  of 
top  managers,  who  operate  in  a world  of  fast  context- switching”  (p.  15).  This  means  that 
short  attention  spans  may  not  ultimately  be  considered  dysfunctional.  Active,  fast-paced, 
high-energy  professionals,  regardless  of  age,  also  fit  this  profile  (e.g.,  CEOs,  scientists, 
researchers,  entrepreneurs,  etc.). 


Implications 

Although  these  comments  would  not  normally  be  included  in  a “demographics”  write  up,  it 
seems  to  me  that  anything  we  can  learn  about  the  college  age  student  helps  us  better  address 
the  teaching  and  learning  context  for  which  we  are  responsible.  It  seems  that  the  Millennials 
are  going  to  be  more  challenging  because  they  have  real  expectations  of  education;  their 
parents  are  also  “more  perfectionist  and  passionate  about  their  kids’  education,”  (p.  147)  and 
that  includes  college.  Parents  began  preparing  these  students  for  college  from  infancy  and  are 
cognizant  about  the  importance  of  homework,  extracurricular  activities,  social  learning. 
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community  service,  and  development  of  every  kind.  Mom  and  Dad  are  curious  about  every 
subject  and  how  it  is  taught;  they  scrutinize  and  provide  (sometimes  undesired)  oversight.  To 
me,  this  is  merely  a factor  in  being  held  accountable.  Having  parents  involved  is  a great 
problem  to  have  rather  than  the  reverse  of  no  involvement.  Having  students  come  in  with 
high  expectations  of  their  educational  experiences  is  another  great  problem.  However,  with 
these  expectations  comes  real  and  time-consuming  work.  We  must  base  teaching  and 
learning  on  the  very  best  research  and  practice;  we  must  strive  to  engage  in  the  Scholarship 
of  Teaching,  engage  in  research  on  teaching  and  learning,  and  share  that  with  our  colleagues. 
In  fact,  we  must  engage  students  with  us  in  the  scholarship  of  teaching. 

Millennial  could  play  an  epic  role,  crafting  new  myths  of  lore,  doing  deeds  only 
dimly  imaginable  today.  The  consequences,  for  good  or  ill,  would  be  enormous — not 
just  for  America,  but  indeed  for  the  entire  world.  Thereafter,  for  the  rest  of  their  lives. 
Millennial  would  collectively  embody  the  transition  into  the  new  modern  order, 
much  as  the  Glorious,  Republican,  Progressive,  and  G.I.  Generations  did  in  prior 
centuries.  ..  Millennials  as  a Hero  Generation... [are  they]  special,  powerful,  capable 
of  great  collective  deeds. . .So  far,  Millennials  are  on  track.  (Howe  & Strauss,  2000, 
pp.  356-60) 

Brown’s  (2000)  findings  about  the  digital  students  can  be  explained  as  dimensional  shifts. 

His  15  year  olds  in  2000  are  just  now  in  college,  our  classrooms,  today.  The  first  is  a literacy 
dimensional  shift  to  more  than  text  and  also  includes  image  and  screen  literacy,  the  ability  to 
‘read’  multimedia  texts,  and  comfort  with  “new  multi-media  genres  that  are  nontrivial”  (p. 
14).  Web  genres  change  quickly;  the  dynamic  of  change  is  great.  Another  literacy  is  one  of 
“navigation. . .the  ability  to  be  your  own  personal  reference  librarian — to  know  how  to 
navigate  through  confusing,  complex  information  spaces  and  feel  comfortable  doing  so. 
‘Navigation’  may  well  be  the  main  form  of  literacy  for  the  2D'  century”  (p.  14).  Yet  another 
dimensional  shift  is  about  learning,  again  “moving  from  authority-based  learning  to 
discovery-based  learning”  and  constantly  discovering  new  things  as  we  browse  through  the 
“emergent  digital  ‘libraries. ’...Web  surfing  fuses  learning  and  entertainment,  creating 
‘infotainment’”  (p.l4).  To  make  the  move  back  to  discovery-based  learning  more  significant. 
Brown  discusses  the  third,  yet  more  subtle,  shift:  forms  of  reasoning.  He  describes  what 
reasoning  has  been,  deductive  and  abstract.  Digital  learners  engage  in  “bricolage...more  than 
abstract  logic.  Bricolage,  a concept  studied  by  Levi-Strauss  more  than  a generation  ago, 
relates  to  the  concrete.  It  has  to  do  with  abilities  to  find  something  - an  object,  tool, 
document,  or  a piece  of  code  - and  to  use  it  to  build  something  you  deem  important. 
Judgment  is  inherently  critical  to  becoming  an  effective  digital  bricoleur”  (p.  14).  This  is 
what  Web-smart  learners  become.  His  final  dimensional  shift  regards  a bias  towards  action. 
The  older  generations  tend  to  want  to  know  how  to  use  something  before  they  try  things  out. 
They  reach  for  a manual,  take  a course,  and  call  an  expert.  Tapscott  (1998)  mentioned  that 
today’s  N-Geners  do  not  know  what  manuals  are;  they  go  to  the  Net.  Brown  described  the 
Web  or  Net-Geners  who 

want  to  turn  the  thing  on,  get  in  there,  muck  around,  and  see  what  works.  Today’s 
kids  get  on  the  Web  and  link,  lurk,  and  watch  how  other  people  are  doing  things,  then 
try  it  themselves ...  .tendency  toward  ‘action’  brings  us  back  into  the  same  loop  in 
which  navigation,  discover,  and  judgment  all  come  into  play  in  5zYm...  learning  in  situ 
with  and  from  each  other. . .learning  is  situated  in  action;  it  becomes  as  much  social  as 
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cognitive,  it  is  concrete  rather  than  abstract,  and  it  becomes  intertwined  with 
judgment  and  exploration,  (p.  14) 


Thus,  the  Web  is  both  an  informational  and  social  resource.  It  has  also  become  a “learning 
medium  where  understandings  are  socially  constructed  and  shared. . .learning  becomes  a part 
of  action  and  knowledge  creation”  (p.  14).  Because  what  Tapscott  (1998)  and  Brown  (2000) 
have  to  say  about  this  is  so  very  fundamental  to  teaching  and  learning  strategies  to  be  used 
with  today’s  students,  it  is  important  to  consider  more  of  what  Brown’s  statement  about  how 
knowledge  is  created  and  shared.  The  two  dimensions  of  knowledge,  according  to  him,  are 
the  explicit  - concepts,  “know-whats.”  The  tacit  is  the  “know-how,”  best  manifested  in  work 
practices  and  skills.  Tacit  knowledge  “lives  in  action. . .comes  alive  through  doing  things,  in 
participation  with  each  other  in  the  world. . . [consequently]  tacit  knowledge  can  be  distributed 
among  people  as  shared  understanding  that  emerges  from  working  together”  (p.  15).  He  goes 
back  to  Bruner’s  observation  (years  ago)  that 

we  can  teach  people  about  a subject  matter. . .its  concepts,  conceptual  frameworks,  its 
facts — and  provide  them  with  explicit  knowledge  of  the  field. . . [but]  being  a physicist 
involves  more  than  getting  all  the  answers  right  at  end  of  the  chapter. . .we  must  also 
learn  the  practices  of  the  field,  the  tacit  knowledge  in  the  community  of  physicist,  that 
has  to  do  with  things  like  what  constitutes  and  ‘interesting’  question,  what  proof  may 
be  ‘good  enough’  or  even  ‘elegant,’  the  right  interplay  between  facts  and  theory- 
formation.  (p.  15) 

This  is  the  difference  between  learning  to  be  a physicist  and  learning  about  physics,  looking 
at  the  explicit  and  tacit,  where  the  “deep  expertise  lies”  (Brown,  2000,  p.  16).  This  means 
that  we  must  learn  the  explicit  knowledge  of  a field,  but  we  must  also  learn  the  “practices  of 
its  community,”  now  referred  to  as  communities  of  practice,  and  the  interaction  between  the 
knowledge  and  practices  of  a field.  This  requires  immersion  into  a community  of  practice, 
being  enculturated  to  understand  the  ways  of  the  community,  the  meanings,  interpretations, 
and  ways  of  acting.  Knowledge  does  not  lie  in  individual  heads  when  the  tacit  (know-how) 
is  factored  into  the  formula.  We  know  much  more  than  the  knowledge  in  our  heads  when  we 
consider  how  to  use  it,  its  practices.  Knowing  is  realized  or  comes  into  being  through 
participation  in  communities  of  practice.  “Learning  to  learn”  occurs  naturally  if  one  is 
situated  in  a community  of  practice,  as  it  has  as  much  to  do  with  understanding  the 
profession  within  which  learning  takes  place  as  it  does  with  facts  and  concepts.  The  Web 
presents  opportunities  for  us  to  “create  a new  fabric  of  learning,”  learning  to  learn  in  situ  - 
the  essence  of  lifelong  learning  (p.  16).  Brown  offers  a great  example  of  copier  technicians 
who  call  each  other  and  reconstruct  stories  of  earlier  fixes,  connect  the  stories  and  figure  out 
together  what  the  problem  is.  They  constructed  their  narrative,  explaining  the  symptoms,  and 
that  resulted  in  the  solution.  It  was  social,  constructive  in  nature,  and  built  on  other  incidents; 
it  was  not  the  usual  logical  reasoning.  Furthermore,  when  they  returned  to  the  office,  they 
swapped  technical  stories,  problems  and  solutions,  building  knowledge  and  solution  banks 
while  drinking  coffee  waiting  for  the  next  calls:  learning  in  situ  - situational  action  oriented 
learning  in  a community  of  practice  from  each  other,  tacit  knowledge  along  side  of  explicit 
knowledge,  cognitively  and  socially  constructed.  As  a result  of  studying  the  technicians,  they 
gave  them  all  radios  that  the  techs  kept  on  so  that  they  could  hear  each  problem  solving. 

Then  if  someone  had  something  to  offer  to  help,  they  could  do  it  instantaneously  because 
they  heard  it;  they  began  to  interact  as  it  was  happening,  while  in  action.  It  worked  extremely 
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well.  This  also  served  well  when  bringing  new  teehnieians  in  [reduced  their  learning  curve]. 
However,  if  some  techs  were  not  involved,  then  they  missed  out.  They  went  to  the  Web  and 
created  a system  to  not  only  record  but  to  validate  the  accuracy  in  stories  across  each  other, 
leading  to  local  heroes  in  their  community  of  practice.  The  Web  provides  a mechanism  to 
capture  and  continue  to  interact  to  build  knowledge  and  learning,  expand  it,  broaden  its 
context,  and  more.  This  is  just  one  example;  he  has  others.  What  is  important  is  that  as 
people  build  knowledge,  they  can  also  construct  the  meanings  and  understandings  of  the 
knowledge  socially.  And  we  can  connect  with  experts  around  the  globe;  we  are  not 
regionally  or  locally  bound.  Cross-pollination  of  ideas  occurs.  Jointly,  individuals  can 
construct  understanding.  The  Net  or  Web  has  transformed  learning  in  our  classrooms.  Digital 
learning  involves  judgment,  navigation,  discernment,  and  synthesis,  which  is  critical 
thinking.  Digital  learners  are  both  a consumer  and  producer  of  learning;  they  “lurk  and 
learn.”  Technology,  to  use  Tapscott’s  (1998)  term,  has  become  “transparent.”  In  Brown’s 
(2000)  terms,  technology  has  shifted  from  supporting  individual  learning  to  supporting 
relationships  between  individuals,  a transformation  of  the  learning  culture.  We  at  four  year 
colleges  and  universities  are  educating  the  leaders  of  the  near  future,  guiding  their 
development  of  ideologies,  values,  and  confidence  - far  more  than  just  knowledge. 

National  Perspectives 

Student  demographics  are  changing.  In  1988,  Atwell  called  for  a new  momentum  for 
minority  participation  in  higher  education  at  the  American  Council  on  Education,  noting  after 
visiting  western  Europe  where  societies  are  small  and  homogeneous  that  while  we  are  the 
most  diverse  nation  and  that  our  success  is  because  of  our  tolerance  and  inherent  pragmatism, 
“our  heterogeneity  has  been  manageable  because  our  size  and  abundant  resources  allow  us  to 
be  expansive.  But  fundamentally,  we  are  a divided  society,  the  fragments  of  which  have  co- 
existed very  well  for  two  centuries  despite  remarkably  little  consensus”  (p.  1).  He  noted  the 
obstacles  that  remained  at  that  time  for  the  new  momentum: 

• Myth  Number  One:  “We  have  tried  that  before  and  it  did  not  work.” 

• Myth  Number  Two:  “Minorities  must  adapt  to  the  institutions  rather  than  the 
institutions  to  minorities.” 

• Myth  Number  Three:  “The  key  is  access;  the  rest  is  up  to  the  individual.” 

• Myth  Number  Eour:  “We  could  do  a better  job  if  elementary  and  secondary  schools 
sent  us  better  prepared  students.” 

• Myth  Number  Eive:  “We  will  never  improve  Black  participation  in  education  until  we 
deal  with  the  deteriorating  social  structures  of  the  Black  community.”  (pp.  1-10) 

Atwell  (1988)  called  for  a new  metaphor,  one  that  reveres  the  integrity  of  each  unique  group 
in  society,  visually  changing  the  image  from  bisque  to  a stew.  He  felt  that  we  were  better  at 
adapting  to  religious  and  ethnic  diversity  stemming  from  Europe  than  to  racial  diversity, 
reminding  us  of  what  we  did  to  the  Native  Americans  and  our  less-willingness  to  accept 
Hispanics.  He  challenged  us  in  many  ways,  especially  our  record  of  success,  which  also 
created  the  malaise.  His  understanding  of  the  context  that  has  now  been  realized  and 
described  herein,  where  America  has  been  “browning,”  has  proven  resilient  and  adaptive 
over  the  past  two  centuries,  and  his  belief  in  our  ability  to  transform  has  born  out.  We  have 
changed,  but  is  it  enough?  Maybe  not! 
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“Colleges  and  universities  are  admitting  the  largest  and  most  diverse  student  population  ever 
known.  The  percentage  of  immigrants  in  our  population  is  as  high,  or  higher,  than  the  end  of 
the  19*  century”  (Lovett,  2003,  p.  35).  CELT,  reflecting  the  national  scene,  has  intentionally 
increased  the  number  of  women  and  minorities  across  its  four  departments.  However,  where 
the  social  compacts  of  the  1960s  and  70s  somewhat  corrected  the  inequalities  between  the 
majority  and  minorities,  there  are  now  other  inequities  to  address  (e.g.  class,  culture,  and 
age).  Since  then  the  college  has  worked  hard  to  create  support  systems  and  mentoring 
programs  and  has  modified  class  schedules  and  made  campuses  more  accessible,  but  today’s 
issues  have  become  more  complex.  The  standards  have  to  change  if  we  are  to  serve  a 
growing  number  of  students  who  are 

25  and  older. . .partially  or  entirely  financially  independent. . .are  combining  college 
attendance  with  other  pursuits. . .attend  more  than  one  institution  before  they 
graduate. . .and  want  opportunities  to  demonstrate  what  they  already  know  and  can  do 
regardless  of  where  they  acquired  that  knowledge.  They  need  to  earn  degrees  based 
not  on  course  grades  but  on  comprehensive  assessments  according  to  criteria  that 
faculty  experts  are  entirely  capable  of  devising  and  implementing.  (Lovett,  2003, 
p.35) 

We  have  a large  international  student  body  within  the  college,  and  more  students  are  working 
part  or  full  time,  have  families  of  their  own,  and/or  have  transferred,  reflecting  our  strong 
community  college  articulation  agreements,  dual  enrollment,  and  positive  attitude  towards 
transfer  students.  Lovett  (2003)  speaks  of  a “new  majority”  of  students  who  could  then  worry 
less  about  attending  a prestigious  institution,  often  exclusive  and  expensive,  or  the  life 
consequences  of  not  attending  such  an  institution  and  focus  more  on  demonstrated 
competencies.  This  would  meet  the  needs  of  the  majority  of  today’s  college  students:  college 
degrees  based  upon  an  assessment  of  competencies  - a results-oriented  approach. 

Lovett  (2003)  goes  on  to  discuss  how  wealth  distribution  and  college  costs  are  affecting 
access  to  higher  education.  A changing  pattern  of  wealth  accumulation,  distribution,  and 
control  in  the  US  as  well  as  wealth  inequalities  (the  income  gap)  have  the  potential  to 
become  an  educational  gap,  which  again  perpetuates  the  income  gap.  The  issues  facing 
higher  education  are  deeper  than  costs  have  gone  up  and  students  cannot  pay.  The  issues  go 
to  the  heart  of  the  arguments  presented  later  about  the  role  of  universities  in  society:  the  role 
and  responsibilities  of  faculties  and  what  students  are  supposed  to  experience.  “Lor  instance, 
the  leaders  of  public  institutions  might  commit  openly  and  unequivocally  to  refocusing 
financial  aid  on  low-  and  middle-income  applicants  instead  of  using  it  for  merit  scholarships 
in  hopes  of  climbing  another  rung  or  two  up  the  ladder  of  academic  prestige,”  e.g.  Carnegie 
classification  (p.  36).  Lovett  discusses  “Interdisciplinarity,  Globalization,  and  Academic 
Structures... [stating  that]  on  most  campuses  the  necessary  transformation  of  academic 
programs  and  organization  practices  to  make  them  more  capable  of  supporting 
interdisciplinary  teaching  and  research  is  still  in  its  infancy. . . .[W]e  [are]  shackled  by 
traditions  and  structures”  (p.  37).  The  obstacles  that  prevent  us  from  engaging  in 
interdisciplinary  scholarship  also  prevent  us  from  shaping  the  nation’s  domestic  agenda  and 
reclaiming  a global  leadership  role. 

This  initiative  is  an  attempt  to  address  some  of  the  underpinning  issues  Lovett  (2003) 
presents.  Brown,  Santiago,  and  Lopez  (2003)  discuss  Latinos  in  higher  education.  Latinos  are 
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the  fastest  growing  ethnic  group  in  the  nation,  with  almost  13%  of  the  population  going  on  to 
college.  About  10%  (over  1.3  million)  of  U.S.  Hispanics  now  have  a college  education,  a 
dramatic  increase  over  the  past  10  years.  The  educational  achievement  of  Latinos  has  become 
very  important  to  higher  education,  as  it  will  impact  the  economic  and  civic  health  of  the 
country.  However,  it  is  important  to  understand  the  challenges.  Most  Latinos  in  higher 
education  are  first-generation  college  students,  low-income,  less  academically  prepared  than 
their  peers,  and  attend  community  colleges  first.  Latinos  are  concentrated  in  a few  states  and 
thus  at  a few  institutions  of  higher  education.  A great  number  of  Latinos  in  higher  education 
are  nontraditional  students,  part-time,  older,  working,  and  with  family  responsibilities.  So 
how  do  we  ensure  their  success?  Through  advocacy  - “As  first-generation  college-goers, 
many  Latino  youth  must  rely  on  formal  sources  of  information  to  tell  them  how  to  prepare 
for  and  participate  in  higher  education”  (p.  42).  With  little  formal  education  and/or  English 
speaking  ability,  their  families  are  limited  in  their  ability  to  help  them  toward  higher 
education.  Often  this  is  misperceived  as  Latino  families  not  valuing  education,  when  in  fact 
just  the  opposite  is  true.  It  is  just  that  Latinos  need  to  know  how  to  navigate  the  system  and 
access  critical  information  about  higher  education.  Many  Latinos  do  not  know  how  to 
overcome  the  financial  issues,  nor  do  they  know  much  about  financial  aid  and  thus  assume 
the  costs  are  beyond  their  ability  to  bear.  They  need  assistance  with  the  admission  processes 
and  especially  need  to  understand  articulation  between  the  community  colleges  and  the 
universities. 

Many  of  these  challenges  exist  for  non-Latino  or  majority  students  as  well;  however,  they 
can  often  navigate  the  application,  financial  aid,  and  other  processes  more  successfully.  Most 
importantly,  “the  underlying  principle  of  institutional  capacity  building  as  a means  of 
enhancing  learning  environments  for  all  students  is  the  basis  for  many  federally  funded 
programs”  and  is  our  responsibility  (Brown  et  ah,  2003,  p.  44).  What  does  that  mean  in  terms 
of  this  particular  initiative?  Good  teaching;  integrated  assessment;  cooperative  and 
collaborative  learning;  learning  opportunities  that  use  a variety  of  intelligences  and  learning 
styles;  engaging  students  in  active,  problem-based,  and  performance-based  learning;  and 
more  are  part  of  the  formula  for  all  students’  success,  and  especially  minorities  and  women. 
This  is  how  we  can  ensure  the  academic  success  of  most,  if  not  all  students;  this  will  also  go 
far  to  ensure  persistence,  retaining  them  until  graduation. 

A point  I have  made  repeatedly  to  professional  development  groups  is  that  the  teaching  and 
learning  strategies  we  are  asking  professors  and  K-12  teachers  to  learn  about  and  use 
improve  the  possibility  of  success  for  minorities  and  women  students;  however,  they  are 
equally  important  for  all  students.  How  often  has  it  been  true,  even  in  honors  classes  or 
classes  for  the  talented,  that  we  do  not  take  those  bright  minds  where  we  could?  In  other 
words,  we  should  be  using  teaching  and  learning  strategies  that  move  those  students  into 
higher  level  learning  activities.  That  is  an  equal  responsibility  for  all  students  regardless  of 
who  they  are  and  what  their  ability  level  is.  If  they  have  been  accepted  into  our  institution 
and  end  up  in  our  classes,  it  is  our  absolute  responsibility  to  challenge,  to  stretch,  and  to 
extend  their  knowledge  and  abilities.  The  point  I want  to  make  is  that  the  good  teaching  and 
learning  strategies,  and  those  of  focus  in  our  initiative,  are  beneficial  to  all  students  and  will 
result  in  increased  knowledge  and  ability  gain.  (See  Literature  sections  throughout)  However, 
because  sometimes  minorities  and  women  in  engineering  and  technology  (and  other  fields) 
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do  not  perform  or  persist  as  well  as  majority  students,  these  strategies  are  especially 
successful  with  these  groups. 


Brown  (2003)  makes  a similar  point  about  institutional  perspective  relating  to  this  point; 
often  “it  is  seen  in  the  institutional  culture,  where  creating  a successful  environment  for 
Latino  students  is  understood  as  a learning  experience  for  all  instead  of  a burden  to  be  borne 
by  some. . .where  leaders  make  sure  that  efforts  to  retain  students  are  not  isolated  but 
systemic. . .hiring  of  new  colleagues  who  value  the  student  population” (p.  44).  My  point 
reflects  this  philosophy  - good  teaching  strategies  are  successful  with  all  students.  An 
especially  good  strategy  is  student  learning  communities.  Our  initiative  described  here  first 
begins  with  trying  to  build  and  develop  a faculty  learning  community  that  will  in  time  move 
towards  developing  formal  student  learning  communities  more  often  throughout  student 
learning  across  courses  and  programs.  (That,  however,  is  yet  to  come;  we  are  building  the 
foundation  first).  One  of  Brown’s  recommendations  supports  my  operational  premise  about 
teaching  and  learning  strategies.  “Promote  the  wider  use  of  proven  strategies  for  helping 
Latino  students  achieve  at  high  levels,  and  develop  better  strategies  based  on  best  practices” 
(p.  46). 


Student  Swirling 

Traditional  students  are  no  longer  the  tradition,  thus  student  demographics  have  great 
implications  for  teaching  and  learning.  College  for  students  is  becoming  a greater  struggle  for 
the  middle  class  and  continues  to  be  difficult  for  the  lower  economic  class.  Although  we  have 
always  had  a large  group  of  students  who  work  their  way  through  college,  the  number  has 
grown.  The  number  of  transfer  students  from  two-year  to  four-year  colleges  is  increasing, 
and  students  may  attend  several  higher  institutions  simultaneously  throughout  the  completion 
of  their  degree  - spiraling.  The  term  “student  swirl”  (along  with  “double  dipping”)  was 
coined  by  Santos  and  Wright  in  1990  to  mean  enrolled  concurrently  in  two  institutions  and 
characterized  by  the  back  and  forth  mode  of  attending  courses  across  both  institutions. 
However,  the  swirl  does  not  only  happen  in  two  year  institutions;  it  also  happens  in  four  year 
institutions  (Adeleman,  1999;  McCormick,  1997  as  cited  in  Borden,  2004).  These  authors 
and  others  also  began  to  focus  on  what  is  know  as  “reverse  transfer,”  transfer  from  four-year 
to  two-year  institutions,  “the  return  trip”  (p.  12).  McCormick  (2003)  differentiated  the  types 
of  swirl  (as  cited  in  Borden): 

1.  trial  enrollment  (experimenting  with  the  possibility  of  transfer  by  taking  a few 
courses) 

2.  special  program  enrollment  (taking  advantage  of  unique  courses  and  programs 
offered  at  other  institutions) 

3.  supplemental  enrollment  (accelerating  progress  by  taking  additional  courses — during 
summer,  for  example — at  another  institution) 

4.  rebounding  enrollment  (alternating  enrollment  at  two  institutions) 

5.  concurrent  enrollment  (that  is,  double  dipping) 

6.  consolidated  enrollment  (taking  a collection  of  courses  at  various  institutions  to 
complete  one  institution’s  degree  program) 

7.  serial  transfer  (one  or  more  intermediate  transfers  on  the  way  to  a final  destination) 

8.  independent  enrollment  (taking  courses  unrelated  to  the  degree  program,  for  personal 
or  professional  interest,  at  other  institutions). 
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There  is  also  an  “intra-institutional  swirl”  in  which  students  enroll  in  travel  abroad,  co-op 
education,  or  exchange  programs,  interruptions  in  a program  sequence  that  can  also  add  time 
to  graduation.  Although  well  known  by  most  administrators,  I believe  relatively  few  faculty 
members  actually  realize  this.  The  implications  can  be  rather  significant  for  students  in  our 
classes.  Students  I know  who  fit  one  or  more  of  the  above  experiences  have  come  to  me  for 
advice  on  scheduling,  to  discuss  credit  for  courses  across  institutions  in  which  segments  of  a 
course  or  whole  course  content  is  similar  or  the  same.  Partial  content  credit  issues  can  be 
complex  and  can  impact  their  expectations  of  our  courses  or  what  they  actually  do  in  our 
courses.  This  “swirling”  situation  needs  to  be  understood  by  our  faculties.  We  benefit  if  we 
know  where  our  students  are  coming  from  and  what  their  backgrounds  include.  Therefore, 
the  participants  in  this  initiative  discussed  the  importance  of  exploring  students’  files  for 
transcripts  during  our  professional  development  program.  The  discussion  was  not  directly 
tied  to  swirling,  but  for  the  purpose  of  better  understanding  the  students,  their  competency 
levels,  and  more.  For  us,  very  important  competencies  are  expected  in  mathematics,  sciences, 
communication  skills,  and  the  prerequisite  or  sequential  aspects  of  a program,  etc. 
Understanding  student  “swirling”  is  also  directly  important;  however,  the  argument  is  that 
professors  are  short  on  time  and  reviewing  student  transcripts  does  take  a good  deal  of  time, 
especially  if  the  courses  enroll  large  numbers  and  swirling  does  not  just  occur  for  general 
education  aspects  of  the  degree  programs  but  involves  courses  in  the  majors  as  well. 

Although  about  general  education,  the  contributors  to  Students  in  the  Balance:  General 
Education  in  the  Research  University  (2002)  make  points  about  instructors  teaching  general 
education  courses  rather  than  professors,  possibly  sending  a message  that  the  courses  are  less 
important  (Keat  & Wright).  These  contributors  go  on  to  challenge  those  general  education 
instructors  on  their  responsibility  for  student  success  and  on  the  failure  of  students  to 
understand  the  curricular  goals  of  a research  university,  but  they  fail  to  discuss  the  goals  of 
general  education  and  how  their  particular  courses  can  help  students  achieve  those  goals. 

The  importance  of  syllabi,  web  pages,  lectures,  and  assignments  do  not  place  [make  sense  of] 
any  one  course  in  the  overall  curricular  structure  for  the  students.  Many  instructors  or 
professors,  themselves,  do  not  understand  the  goals  of  the  general  education  program,  for  it  is 
acknowledged  and  supported  as  program  rather  than  a series  of  preparatory  courses  in  most 
progressive  universities  today.  When  considering  the  lack  of  interest  or  motivation  for 
faculty  members,  it  is  important  to  remember  that  faculty  members  are  truly  required  to 
specialize  in  their  fields  if  they  are  to  be  awarded  tenure  and  promotion,  thus  they  are  not 
usually  focused  on  general  education.  Faculty  members  have  few  incentives  to  want  to  teach 
general  education  courses.  Thus  teaching  general  education  has  to  be  addressed  by  the 
administration  of  all  departments  to  make  it  important.  After  all,  general  education  prepares 
students  for  advanced  courses,  and  if  we  expect  them  to  come  prepared,  it  is  important  that 
the  students  tie  the  knowledge  and  skills  learned  in  those  courses  to  the  courses  in  the  majors. 
For  us,  this  discussion  about  general  education  has  importance.  Our  faculty  development 
program  included  an  analysis  process  to  itemize  the  general  education  knowledge  and  .skills 
embedded  within  the  major  knowledge  content,  showing  the  relationship  between  general 
education,  accreditation  criteria  or  standards,  and  the  student  learning  objectives  and  course 
content.  This  linking  has  been  received  very  well  by  the  program  leaders’  students,  thus  those 
in  the  program  are  trying  a new  syllabus  structure  and  content  in  their  revised  courses. 
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When  specifically  considering  women  and  men  who  have  chosen  engineering,  possibly  with 
implications  for  engineering  technology  (accredited  by  ABET  and  somewhat  for  the  field 
identified  as  technology,  accredited  by  NAIT),  we  reviewed  the  collaborative  study  by  the 
U.S.  Department  of  Education  and  the  National  Science  Eoundation  (1998).  The  study’s 
focus  was  the  paths  students  take  through  higher  education,  with  engineering  as  the  case, 
because  it  brings  all  the  variables  affecting  choice,  persistence,  and  migration  into  play  and  is 
a field  where  there  has  been  an  historically  severe  gender  imbalance.  The  study  used  student 
transcripts  from  1982-1993  and  the  strategy  of  “the  student  as  the  story-maker,”  a somewhat 
different  than  traditional  approach.  Selected  findings  regarding  the  gender  and  classroom 
aspects  only  are  that 

• Women  who  intended  to  major  in  engineering  enjoyed  the  highest  degree  of  parental 
support  for  bachelor’s  degree  attainment  among  all  women  - or  men  - who  intended 
to  major  in  any  field. 

• Women  and  men  earn  similar  grades  in  engineering  courses,  and  the  women  who 
leave  engineering  have  higher  grades  than  the  men  who  leave. 

• Women  who  leave  engineering  do  not  leave  because  of  poor  academic  performance, 
though  they  do  evidence  a higher  degree  of  academic  dissatisfaction,  (p.  xi) 

When  experiencing  engineering  education,  classrooms,  credit  loads,  and  grades,  the  U.S. 
Department  of  Education  and  the  National  Science  Eoundation  (1998)  acknowledged  that 
we  have  not  been  tracking  students  in  higher  education  very  well  in  terms  of  initial 
field  choice  and  change  of  major  as  they  search  for  academic  identity;  [this] 
traffic. . .moves  at  a high  rate;  provosts  and  deans  worry  about  this  because  it  affects 
their  ability  to  plan;  and  some  fields  worry  about  attrition  and  migration  because  they 
have  historically  exhibited  equity  problems,  (p.  xii) 

Thus,  the  focus  is  on  engineering  because  the  literature  indicates  its  enrollments  are  volatile, 
attrition  is  reported  as  very  high,  and  it  is  considered  a highly  gender-segmented  field. 

“. . .women  who  begin  the  study  of  engineering  in  college  are  less  likely  to  complete  a degree 
in  engineering  than  men,  thus  exacerbating  the  segmentation”  that  begins  in  high  school 
where  students  acquire  “curricular  momentum”  and  where  a “higher  proportion  of  women 
than  men  have  that  curricular  momentum  but  do  not  choose  to  explore  even  the  threshold  of 
the  engineering  path  in  college”  (p.  81).  Those  writing  the  monograph  concluded  after 
studying  the  works  of  others  (e.g.  Becher,  1989;  Chelapati,  1990;  Constantinople  et  ah,  1988; 
Davis  et  ah,  1995;  Eelder  et  ah,  1995;  Grandy,  1994;  Henes  et  ah,  1995;  Serex,  1997; 
Seymour  & Hewitt,  1997)  that 

• Students  who  complete  engineering  degrees  are  no  more  dissatisfied  with  their 
college  experience  than  non-engineering  students  (p.  82). 

• Student  choice  is  an  evolving  phenomenon,  and  it  is  not  very  accurate  to  talk  about 
“attrition”  in  any  field  until  a student  actually  starts  to  major  in  it,  then  leaves  for 
something  else.  Women  and  men  who  leave  engineering  are  more  likely  take  their 
curricular  momentum  into  computer  science  and  the  physical  sciences  than  other 
majors  (and  in  our  direct  case.  Engineering  Technology  or  Technology);  and  women 
who  leave  the  engineering  path  are  more  likely  to  complete  bachelor’s  degrees  than 
are  men.  (pp.  83-84) 

• Ereshmen,  regardless  of  disciplines,  possess  a similar  range  of  concepts  and  are 
equally  as  likely  to  attribute  the  broad  scientifically  based  problems  to  both  social 
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behavior  and  technological  developments.  Engineering  students,  however,  were  more 
confident  in  technology  as  a solution. . .one  of  the  reasons  they  are  majoring  in 
engineering!  (p.85) 

• There  are  further  implications  [serious  challenges]  of  a different  vision  of  recruitment 
for  the  undergraduate  curriculum  because  engineering  students  face  a daunting  set  of 
requirements  in  terms  of  credit-load.  Engineering  education  faces  a tension  between 
superficial  coverage  (“a  sense”)  and  the  additional  credits  and  time  that  come  with 
depth.  They  suggest  changing  the  degree  for  today’s  times  and  needs,  (p.  86) 

• The  strategy  of  lower-division  science  courses,  Tobias  (1990  as  cited  in  NSE,  1998) 
argues,  should  be  to  cultivate  not  weed  (though  “weed  out”  may  not  be  the  most 
felicitous  of  terms  to  describe  what  happens),  and  these  courses  are  still  part  and 
parcel  of  new  program  combinations.  No  discipline  can  maintain  enrollment  shares 
with  a weed-out  system,  and  yet  each  discipline  has  a culture  that  naturally  diverts 
some  students  onto  other  paths. 

• Yes,  we  can  improve  the  way  science  and  engineering  are  taught,  particularly  in  large 
institutions,  but  let  us  not  pretend  that  these  are  the  only  domains  in  which  such  an 
effort  is  necessary. 

• Men  have  been  a distinct  and  declining  minority  in  undergraduate  education 
(enrollees  and  degree  recipients)  for  more  than  a decade,  (p.  87) 

Kuh  (2003)  reports  on  the  National  Survey  of  Student  Engagement  (NSSE)  results.  This 
survey  is  an  effort  to  improve  the  assessment  of  the  quality  of  undergraduate  education  by 
looking  closely  at  the  level  of  student  engagement  during  the  academic  career.  The  survey 
focuses  specifically  on  educational  activities  related  to  learning  and  personal  development. 
The  NSSE  Benchmarks  are  (1)  Eevel  of  Academic  Challenge;  (2)  Active  and  Collaborative 
Eearning;  (3)  Student-Eaculty  Interaction;  (4)  Enriching  Educational  Experiences;  and  (5) 
Supportive  Campus  Environment.  Three  years  of  data  on  600  four-year  colleges  and  285,000 
first  year  and  senior  students  reveal,  among  many  other  things,  that  the  following  are  more 
“engaged”: 

• Women 

• Eull  time  students 

• Students  who  live  on  campus 

• Native  students  (those  who  start  and  graduate  from  the  same  school) 

• Eearning  community  students 

• Students  with  diversity  experiences  (p.  26) 

Eor  the  most  part,  Kuh  (2003)  explains  that  this  is  to  be  expected  with  full  time  students  who 
live  on  campus,  commenting  that  these  students  would  probably  have  fewer  obligations, 
family  responsibilities,  and  off-campus  work  that  would  keep  them  from  participating  in  or 
accessing  opportunities  and  resources.  He  did  not  comment  on  women  in  particular  as  to  why 
they  were  more  engaged;  however,  he  did  go  further  to  say  students  of  color  “experience 
college  differently  than  white  students”  but  noted  that  they  “engage”  at  a comparable  level 
(p.  26  ).  African-American  students,  however,  report  lower  grades  (with  white  students 
getting  the  highest  grades),  even  though  the  GPA  is  positively  related  as  the  same  for  all 
benchmark  scores  and  educational  practices. 
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On  the  question:  Are  students  putting  forth  enough  academic  effort,  Kuh  (2003)  discusses 
that  life  situations  of  both  traditional-age  and  returning  college  students  are  more 
complicated.  Life  exigencies  such  as  full  time  work  and  care  for  dependents  seem  to  limit  the 
amount  of  time  that  can  be  spent  on  studies.  He  especially  noted  that  “Even  the  majority  of 
traditional-aged,  full-time  students  are  working  by  the  time  they  are  seniors”  (p.  27). 

Students,  however,  expect  to  be  more  engaged  than  they  are  (e.g.  read  more,  write  more,  and 
become  involved  in  cultural  activities).  They  expect  to  and  actually  do  study  more  than  in 
high  school,  but  faculty  feel  more  is  needed.  Of  the  academe’s  expectation  “two  for  one” 
(study  two  hours  for  each  hour  of  class),  students  only  spend  about  half  of  that.  Students  are 
coming  to  class  unprepared,  saying  that  there  is  little  emphasis  on  studying  and  spending 
time  on  academic  work.  Thus  when  students  are  not  engaged  and  spend  less  time  on  studies, 
they  do  not  perform  well.  Kuh  mentions  the  problem  partially  begins  before  college  when 
students  are  not  engaged  in  high  school  but  graduate  with  A’s,  thus  implying  that  higher 
grades  are  received  for  less  work  or  lower  performance. 

Kuh  (2003)  poses  the  question  “Are  we  willing  to  make  the  effort  that  such  practices  demand 
of  us?”  Students  rarely  exceed  their  own  expectations  about  academic  work.  Conversely, 
students  will  go  beyond  what  they  think  they  can  do  if  conditions  exist  where  their  teachers 
expect,  challenge,  and  support  them  to  do  so.  When  we  demand  that  they  do  something,  they 
will  do  it.  When  prompt  feedback  is  provided,  they  learn  more.  The  more  students  are  asked 
to  do,  the  more  we,  as  professors,  have  to  do  to  prepare  the  activities  and  give  feedback  to 
students.  He  says  there  is  a direct  relationship  between  what  we  do  and  student  response  - 
the  more  we  do,  the  more  likely  students  will  visit  during  office  hours  to  talk  about  the 
feedback.  This  sometimes  creates  a dilemma  in  faculty  time  allocation  to  multiple  priorities. 
Often,  a “disengagement  compact”  occurs  in  which  “I’ll  leave  you  alone  if  you  leave  me 
alone.  I won’t  make  you  work  too  hard  (read  a lot,  write  a lot)  so  I won’t  have  to  grade  as 
many  papers  or  explain  why  you  are  not  performing  well”  (p.  28).  Kuh  feels  the  evidence  of 
the  existence  of  the  pact  is  that  many  students  get  decent  grades,  B’s  and  sometimes  better, 
for  a low  level  of  effort.  He  feels  this  is  a breakdown  of  shared  responsibility  where  faculty 
allow  students  to  get  by  with  less  than  maximal  effort  and  where  students  are  not  taking 
advantage  of  available  resources.  If  college  is  to  be  transforming,  there  is  no  substitute  for 
“time  on  task”;  it  is  a “once-in-a-lifetime  opportunity  to  challenge  students  to  examine 
previous  ways  of  knowing,  thinking,  and  behaving”  (p.  28).  If  students  do  not  engage  enough 
to  develop  the  habits  of  mind  and  heart  of  an  educated  person,  then  that  opportunity  is 
missed. 

When  considering  the  question  “Is  the  active  and  collaborative  learning  movement 
inadvertently  undercutting  academic  effort?”  - it  seems  that  faculty  members  are  responding 
by  structuring  active  and  collaborative  learning  activities  based  upon  reports  from  students. 
But  there  is  no  information  that  can  judge  the  quality  of  these  learning  experiences.  The 
numbers  only  reflect  the  frequency  students  report  being  involved  in  these  types  of 
experiences.  Some  anecdotal  reports  indicated  that  students  may  not  prepare  as  much  for 
classes  in  which  in-class  group  work  occurs,  relying  on  their  group  members  to  make  it  all 
happen.  Well-designed  and  executed  collaborative  group  work  that  is  active  would  prevent 
this  by  building  in  individual  and  group  accountability,  etc. 
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“How  much  faculty  interaction  is  enough”  is  not  an  easy  question  to  answer;  the  key  is 
substantive  contact.  Kuh  (2003)  reports  that  casual  contact  does  little  to  increase  learning 
gains  or  effort.  If  technology  is  used  effectively,  it  seems  that  “student  success  can  be 
achieved  in  class  without  increased  student-faculty  contact”  (p.  29),  but  it  requires  faculty  to 
be  more  intentional  and  available  on  an  as  needed  basis.  Sometimes  occasional  contact  is 
enough.  The  nature  and  frequency  of  contact  matter,  but  there  are  six  purposes  for  contact 
between  students  and  faculty:  (1)  career  planning;  (2)  working  on  a committee,  project;  (3)  or 
doing  research  together.  The  first  two  might  only  need  to  occur  once  or  twice  per  semester. 
However,  working  on  research  with  a faculty  member  could  alter  a student’s  life.  The  next 
three  are  more  critical  in  that  they  should  occur  more  often:  (4)  prompt  feedback  on 
performance;  (5)  discussing  grades  and  assignments;  and  (6)  discussing  ideas  outside  of 
class. 

Kuh  (2003)  reports  that  transfer  students  are  usually  less  engaged.  The  study  reveals  that  at 
some  universities,  the  number  of  transfer  students  exceeds  70  percent  (the  CEET  percentage 
is  approximately  40%).  The  NSSE  concluded  that  “transfers  generally  find  their  institutions 
as  academically  challenging  as  their  non-transfer  peers  do,. . .report[ing]  comparable  grades, 
and  are  more  likely  to  be  prepared  for  class  than  non-transfer  students”  (p.  32  ).  This  could 
be  that  they  are  older  and  commuting,  and  more  than  50%  are  first-generation  students.  There 
may  be  many  other  reasons  for  less  engagement  (e.g.  new  institution,  different  culture,  and 
transfers  do  not  have  specially  designed  socializing  experiences  such  as  the  ones  for 
incoming  freshmen).  Thus,  this  is  a challenge  leading  to  questions  about  articulation, 
performance  indicators,  and  more.  Eess  engagement  by  transfers  does  not  seem  to  be  a 
function  of  attending  a certain  type  of  institution,  meaning  transfers  from  community 
colleges  to  university  versus  four  year  to  four  year  college  transfers.  Eess  engagement 
prevails  regardless.  The  two-year  college  sector  is  growing,  and  more  students  are  attending 
multiple  institutions  to  achieve  a baccalaureate  degree.  We  can  expect  continued  increases; 
therefore,  it  is  important  to  find  ways  to  more  fully  engage  transfer  students  through  more 
effective  educational  practices. 

As  for  “Does  experience  with  diversity  matter  to  student  engagement?”  - knowing  how  to 
effectively  work  with  diverse  individuals  is  critical.  Students  reporting  more  experience  with 
diversity  are  more  involved  in  other  effective  educational  practices,  e.g.  active  and 
collaborative  learning.  There  is  still  a need  for  schools  to  look  for  ways  to  ensure  that 
students  explore  human  differences  in  positive  and  purposeful  ways,  for  “more  than  a fifth  of 
all  seniors  think  that  their  schools  give  little  emphasis  to  encouraging  contact  between 
students  from  different  economic,  social,  and  racial  [ethnic]  backgrounds”  (p.  34). 

Eight  and  Cox  (2001)  spent  10  years  systematically  researching  and  exploring  how  to 
facilitate  the  best  possible  undergraduate  experience.  They  present  a synthesis  of  the  work 
formed  around  two  primary  questions:  “first,  what  choices  can  students  themselves  make  to 
get  the  most  out  of  college?”  and  “second,  what  are  effective  ways  for  faculty  members  and 
campus  leaders  to  translate  good  intentions  into  practice?”  (p.  3).  All  findings  come  directly 
from  intense  student  interviews  with  more  than  1600  undergraduates,  some  more  than  once. 
One  of  the  most  important  insights  is  that  learning  outside  the  classroom  is  vital,  for  example 
in  residential  settings  or  through  extracurricular  activities;  students  reported  that  outside 
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classroom  events  were  where  they  experienced  the  most  profound  changes.  Another  finding, 
important  to  our  initiative,  is  that  students  reported 

they  learn  significantly  more  in  courses  that  are  highly  structured,  with  relative 
quizzes  and  short  assignments. . . .Crucial  to  this  preference  is  getting  quick 
feedback  from  the  professor  - ideally  with  an  opportunity  to  revise  and  make 
changes  before  receiving  a final  grade.  In  contrast,  students  are  frustrated  and 
disappointed  with  classes  that  require  only  a final  paper.  How  can  we  ever  improve 
our  work,  they  ask,  when  the  only  feedback  comes  after  a course  is  over  and  when  no 
revision  is  invited?”  (p.  9) 

Yet  another  surprise  was  about  homework  and  the  controversy  regarding  “collaboration,” 
historically  considered  cheating.  Today  professors  encourage  students  to  collaborate  or  work 
together,  often  creating  small  study  groups  where  students  work  together  outside  of  class.  To 
us  this  is  the  foundation  for  student  learning  communities.  Sometimes  the  academic 
assignments  are  so  challenging  or  complex  that  the  only  way  to  get  through  them  is  to 
collaborate,  and  many  students  report  that  such  homework  assignments  increase  their 
learning  and  their  engagement  in  class  (Light  & Cox,  2001,  p.  9).  This  directly  supports  our 
initiative  but  is  still  a new  format  for  learning  on  campuses.  Furthermore,  students 
particularly  value  a faculty  “mentored  internship  - not  done  for  academic  credit”  in  which 
the  student  engages  in  a project  under  faculty  supervision,  using  their  own  plan  (not  the 
faculty  member’s)  on  a topic  they  care  about  (p.  9). 

Light  and  Cox  (2001)  go  on  to  report  that  although  students  are  perceived  as  not  interested  in 
the  sciences  and  engineering  and  the  faculty  members  are  interested  in  research,  this  is  not 
really  true.  More  male  and  female  students  are  strongly  interested  in  the  fields  than  any  other 
group  of  courses  and  they  would  not  want  to  work  with  faculty  not  actively  pursuing  their 
own  research.  A student  said 

Of  course,  I want  a faculty  advisor  who  is  a good  teacher,  who  is  kind,  who  is  willing 
to  spend  time  supervising  me,  who  is  available,  patient,  and  who  explains  things 
clearly.  But  if  to  get  that  level  of  perfection  I had  to  work  with  a faculty  member  who 
was  not  actively  doing  research,  I am  not  sure  why  I would  seek  such  a person  to 
supervise  me.  My  goal  is  not  just  to  learn  biology.  It  is  much  more  than  that, 
especially  by  junior  and  senior  year.  It  is  to  learn  how  to  really  do  biology.  And  it 
seems  pretty  clear  to  me  that  to  do  biology  I need  to  learn  from  someone  who  is 
actually  doing  it  too.  (p.  71) 

Going  further.  Light  and  Cox  (2001)  present  another  perception  that  many  undergraduates 
avoid  the  physical  sciences  because  they  feel  they  cannot  do  the  work  or  that  they  avoid 
science  classes  because  the  workload  is  much  heavier;  this  is  also  wrong.  They  find  that 
about  30%  express  these  concerns  about  their  preparation  to  do  well  in  such  courses.  The 
other  70%  choose  other  disciplines  because  of  other  reasons.  The  perception  of  heavier  work 
loads  is  partially  correct  and  incorrect;  natural  sciences  do  have  heavy  loads  but  are  usually 
tied  with  language  classes  in  workload.  Their  work  load  is  rated  just  slightly  higher  than 
those  for  the  humanities  and  social  sciences,  so  this  does  not  bear  out  as  well.  Finally,  a 
perception  that  there  is  more  grade  competition  among  students  in  the  sciences  is  clearly 
correct.  In  science  classes,  the  competition  for  grades  is  well  above  other  disciplines.  Most 
important  to  our  initiative  is  that  the  students  said  that  in  small  classes  they  can  get  to  know 
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the  professor  better,  one  on  one;  and,  the  professor  can  use  teaching  techniques  that  are  more 
difficult  to  use  in  larger  classes  (e.g.  organizing  the  class  around  a controversy,  presenting 
the  opportunity  to  develop  arguments  and  argument  skills).  Light  and  Cox  make 
recommendations  for  faculty  consideration  about  workload  and  grade  competition,  study 
groups,  increasing  interaction,  and  how  to  attract  and  keep  more  students.  Most  of  their 
suggestions  were  addressed  in  our  professional  development  on  teaching  and  learning.  They 
confirm  our  directions. 

Light  and  Cox  (2001)  also  learned  that 

how  students  study  and  do  their  homework  is  a far  stronger  predictor  of  engagement 
and  learning  than  particular  details  of  their  instructor’s  teaching  style;  thus,  the  design 
of  homework  really  matters. . .specifically,  those  students  who  study  outside  of  class 
in  small  groups  of  four  to  six. . .benefit  enormously;  they  do  homework  independently 
before  they  meet.  The  meetings  are  organized  around  discussions  of  homework.  And 
as  a result  of  their  study  group  discussion,  they  are  far  more  engaged  and  far  better 
prepared,  and  they  learn  significantly  more.  (p.  52) 

Also  an  overwhelming  number  of  interviewed  students  felt  that  “the  impact  of  racial  and 
ethnic  diversity  on  their  college  experience  is  strong. . . [and]  an  overwhelming  majority  of 
undergraduates  characterize  its  effects  as  highly  positive,”  but  they  noted  learning  from 
others  who  are  different  does  not  always  happen  naturally,  thus  campus  atmosphere  and 
living  arrangements  are  crucial. . .also  noting  “only  when  certain  preconditions  are  met  does 
‘the  good  stuff’  actually  happen. . .that  those  preconditions  are  factors  that  campus  leaders 
can  do  something  about. . .shap[ing]  an  environment  in  which  diversity  strengthens  learning.” 
(Light  & Cox,  2001,  p.  9). 

These  concepts  go  to  the  heart  of  what  we  are  trying  to  change  as  a result  of  our  initiative.  If 
both  individual  and  group  accountability  are  structured  into  group  work  and  the  assignments 
have  integrity  and  are  based  on  sound  criteria  with  well  designed  rubrics,  work  outside  the 
classroom  can  be  high  quality.  However,  this  philosophy  and  the  skill  set  and  time  that  are 
required  to  begin  to  learn  to  use  highly  structured,  but  open  ended  and  accountable,  group 
work  take  time  to  develop,  at  least  at  first. 

Another  result  in  Light  and  Cox’s  (2001)  study  was  about  writing.  Most  surprisingly  is 
students  feel  very  strongly  about  good  writing  and  are  greatly  concerned  about  their  writing 
and  want  to  develop  good  writing  skills.  Many  professors  today  do  not  require  writing 
assignments,  as  it  takes  extensive  time  to  grade  them.  Therefore,  students  do  not  get  the 
opportunity  to  practice  and  improve  across  their  courses.  We  have  professors  who  feel  that 
once  students  reach  their  major  they  should  not  be  writing  instructors  and  should  only  grade 
for  content,  although  some,  of  course,  understand  that  we  are  both.  Light  and  Cox  describe 
students  who  improve  the  most  as  those  who  worked  on  writing  very  intensely  with  a 
professor  or  writing  teacher  or  small  group  of  fellow  students  and  that  the  longer  they 
worked  on  it,  the  greater  the  improvement.  This  professor  believes  in  many  written 
assignments  with  a mix  of  types  of  written  assignments  and  requires  the  Writing  Center  visits 
for  developing  the  ideas  and  then  for  critiquing  the  writing  itself.  When  my  students  get  their 
written  assignments  back,  there  are  several  colors  of  comments;  some  students  commented 
that  they  could  not  believe  the  amount  of  work  evident  in  my  feedback.  They  let  me  know 
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they  were  impressed  that  I had  worked  that  hard  on  their  work.  It  makes  sense,  however, 
because  the  more  writing,  the  more  group  work,  and  the  more  time  students  have  to  spend  on 
their  courses,  the  more  their  level  of  engagement  increased.  Although  no  single  course,  even 
with  many  writing  assignments,  can  correct  poor  writing  habits,  several  courses  that 
emphasize  writing  can  at  least  make  students  aware  of  their  writing  issues  so  they  know 
when  to  have  it  edited.  Also  when  students  have  to  write,  they  are  really  engaged.  There  is  no 
way  to  escape  being  engaged  in  course  content  and/or  the  class  itself  if  exchanges  are 
executed  through  jigsaw,  etc.  We  have  introduced  that  to  the  professors  involved  in  our 
initiative,  and  I believe  some  have  taken  new  steps  in  regard  to  writing.  However,  it  is  very 
important  to  note  that  the  structure  and  meaning  of  the  written  assignments  is  extremely 
important.  Just  as  for  group  work,  there  has  to  be  serious  purpose;  specific  criteria  for 
content,  argument,  and  mechanics;  a good  rubric  for  scoring;  etc.  for  it  to  be  meaningful  and 
not  just  busy  work.  Some  of  us  use  the  “one  minute  paper”  that  Light  and  Cox  mention,  and 
some  even  before  the  idea  was  published  as  the  “one  minute  paper”  by  Felder  (1988)  and 
Johnson,  Johnson,  and  Smith  (1998).  It  is  a good  teaching  and  learning  strategy  and  has 
varied  methods  of  use. 

Finally,  Light  and  Cox  (2001)  go  on  to  summarize  several  other  findings:  students  who  get 
the  most  out  of  college  include  activities  with  faculty  members  or  several  other  students  to 
accomplish  substantive  academic  work.  They  “hungered  for  specific  suggestions  about  how 
to  improve  it”  (p.  1 1).  Students  went  on  to  describe  particular  activities  outside  the  classroom 
as  profoundly  affecting  their  academic  performance  (e.g.,  time  management  activities,  small 
group  work,  and  study  techniques).  Students  discuss  “foreign  language”  or  other  such 
requirements,  literatures,  with  enthusiasm;  many  giving  them  the  highest  ratings  - it  is  all  in 
how  they  are  taught  and  organized.  Light  and  Cox  conclude,  from  analyzing  1600  student 
interview  responses,  that  students  give  a lot  of  thought  to  what  works  well  for  them  (in  an 
academic  environment)  and  can  offer  their  insights  to  learn  how  to  improve  it.  The  responses 
were  fairly  consistent  whether  with  Harvard  or  other  national,  regional,  small,  large,  private, 
or  public  institutions. 

Light  and  Cox’s  (2001)  results  directly  support  the  Scholarship  of  Teaching  arguments 
presented  by  Boyer  (1990)  and  others.  The  results  also  support  the  intentions  of  our  initiative 
and  how  to  engage  students,  regardless  of  their  demographics.  Our  initiative  begins  with 
trying  to  build  and  develop  a faculty  learning  community  that  will  in  time  move  toward 
developing  formal  student  learning  communities  throughout  student  learning  across  courses 
and  programs.  (That,  however,  is  yet  to  come;  we  are  building  the  foundation  first.) 
Therefore,  we  offer  thoughts  and  some  literature  on  learning  communities.  Additional 
information  can  be  found  in  the  literature  review. 

Need  for  Student  Learning  Communities  (LC) 

Although  our  primary  focus  for  this  initiative  is  to  create  and  sustain  faculty  learning 
communities,  we  hope  to  evolve  further  to  where  the  work  with  our  professors  will  result  in 
creating  formal  student  learning  communities.  Understanding  our  students  is  fundamental  to 
that  purpose.  Therefore,  the  demographic  information  can  go  far  to  assist  our  faculty 
members  in  their  desire  to  better  understand  their  students  and  to  create  a learning 
environment  that  will  stimulate  and  motivate  them  to  achieve  to  their  potential. 
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Schroeder  (1993)  found  that  more  than  half  of  today’s  students  perform  best  in  a learning 
situation  characterized  by  “direct,  concrete  experience,  moderate-to-high  degrees  of 
structure,  and  a linear  approach  to  learning”  (as  cited  in  Levine  & Cureton,  1998,  p.  25). 

They  value  the  practical  and  the  immediate,  and  the  focus  of  their  perception  is  primarily  on 
the  physical  world.  However,  75%  of  faculty  members  “prefer  the  global  to  the  particular, 
are  stimulated  by  the  realm  of  concepts,  ideas,  and  abstractions  and  assume  that  students,  like 
themselves,  need  a high  degree  of  autonomy  in  their  work”  (p.  25).  Students  prefer  the 
concrete  subjects  and  active  methodology  of  learning.  However,  students  are  highly  satisfied 
with  higher  education  and  would  not  change  it  much,  even  though  they  do  not  believe  it 
guarantees  them  a good  job.  However,  they  understand  that  they  cannot  get  one  without  it.  It 
is  discrepancies  such  as  these  that  influence  this  initiative. 

For  example,  in  the  undergraduate  courses  taught  by  me  over  the  years  and,  more  recently, 
the  senior  design  capstone  course,  my  reputation  has  been  known  as  "the  witch  [professor] 
from  hell."  And  when  students  evaluate  the  course,  my  experience  (although  only  once  and  a 
long  time  ago)  was  a reduced  score  on  the  course  evaluations  because  of  the  required  level  of 
active  learning  strategies  that  students  had  not  experienced  before  my  course.  To  explain  this, 
and  without  ego  I hope,  in  the  undergraduate  senior  capstone  course,  students  come  into  my 
course(s)  having  heard  about  the  demanding  nature  of  the  requirements  and  activities, 
meaning  the  expectations  for  both  individual  and  team  assignments,  formal  teaming,  extra 
meeting  times,  and  very  specific  outcomes.  It  was  interesting  to  note,  however,  that  within 
three  to  four  weeks  (or  sooner),  they  began  to  see  relevance,  validation,  and  the  important 
connections  to  both  the  theoretical  and  conceptual  content  that  is  so  very  dear  to  professors, 
but  beyond  that,  they  began  to  see  its  connection  to  the  external  world  as  a result  of  "how" 
they  learn  this  content.  They  began  to  realize  the  importance  of  the  “formal”  team 
requirements  as  well  as  understand  the  individual  accountability  built  into  the  course  by  the 
individual  assignments  required  to  prepare  them  for  the  team  experience.  In  addition,  they 
began  to  realize,  respect,  and  appreciate  my  direct  focus  and  began  to  understand  why  we  did 
not  make  some  of  the  changes  a few  of  them  may  have  requested  to  make  the  course  fit  better 
into  their  overall  lives. 

When  I require  that  they  function  in  true  teams,  this  means  they  are  formally  trained  to  be 
teams  - not  just  assigned  to  working  groups.  They  have  to  fully  execute  a technical  project 
according  to  very  specific  criteria  and  outcomes  for  the  team,  project,  course  content,  etc.  If 
their  teams  had  difficulty  (and  we  tried  to  ensure  that  every  team  had  difficulty),  then  some 
of  them,  although  very  rarely,  did  not  see  beyond  the  immersion  experience  to  what  they  had 
learned  (even  though  we  thoroughly  explain  that  each  team  must  experience  difficulties),  as 
that  was  a requirement  to  work  though  the  team's  learning  content  and  process.  For  others 
who  might  try  these  teaching  and  learning  strategies,  team  members’  interpersonal 
difficulties  may  translate  into  a reduced  overall  course  evaluation  score.  However, 
simultaneously,  in  their  formal  and  very  analytical  team  presentations,  they  all  actually  raved 
positively  about  what  they  learned,  illuminating  with  examples,  content,  products,  and 
processes  - all  a professor  could  ever  hope  for  from  students.  The  point  here  is  that 
professors  must  be  capable  of  risking  their  end  of  course  evaluation  "score"  if  they  are 
fundamentally  and  primarily  concerned  with  the  integrity  and  rigor  of  the  course,  teaching, 
and  student  learning  experiences.  This  is  a great  leap  for  many. 
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Although  we  felt  that  this  initiative  had  to  first  begin  with  developing  a faculty  learning 
community,  do  not  assume  that  there  are  no  student  learning  communities  throughout  our 
departments,  courses,  or  curricula.  Although  few,  some  professors  have  very  active  informal 
student  learning  communities  engaged  in  complex  problem  solving  through  real  world  or 
simulated  problems/projects  where  student  groups  (not  really  teams  or  formal)  are  structured 
to  work  together.  It  was  our  goal  to  create  a formal  faculty  learning  community,  and  by 
modeling  that  design,  process,  and  procedures,  they  would  see  the  importance  of  the 
“formality”  of  structuring  student  teams,  ultimately  learning  communities.  If  formal,  the 
outcomes  could  better  be  required  and  measured.  Professors  must  determine  if  they  are 
measuring  only  discipline- specific  knowledge  and  skill  gain  and/or  if  they  are  going  to 
measure  student  performance  in  teams  and  the  community  as  part  of  the  course  knowledge 
and  skills.  This  determination  is  critical,  as  it  goes  beyond  what  is  perceived  as  “typical” 
disciplinary  content.  However,  team  experience  and  performance  is  today  one  of  the 
accreditation  program  outcomes,  so  it  is  legitimate  as  “course  knowledge  and  skills.”  If  it  is 
to  be  included  and  measured,  then  there  must  be  established  team  and  community 
performance  criteria  for  students  to  understand  the  interactive  processes  and  dynamics  and 
differences  between  working  individually  versus  in  teams  and  as  teams  within  a larger 
community  of  teams.  When  knowledge  and  skill  learning  is  critical  to  the  success  of  the 
student  individually,  then  individual  and  team/community  accountability  must  be  ensured 
through  the  teaching  and  learning  process.  Also  if  individual  success  is  measured  by  his/her 
cooperation  and  collaboration  with  other  team  members,  it  also  must  be  established  in  the 
performance  criteria.  Extending  the  point  of  individual  versus  group  “accountability,”  it  is 
important  for  the  professor  to  determine  what  is  acceptable  performance  or  evidence  of 
learning  disciplinary  content.  Does  each  student  need  to  provide  evidence  of  learning  or  is  it 
acceptable  for  the  group  to  provide  a collective  evidence  of  learning?  This  is  mentioned  by 
Johnson,  Johnson,  and  Smith  (1998)  as  building  in  “individual  accountability”  into  group 
work  and  is  critical  when  structuring  collaboration  or  cooperative  learning  for  teams  and/or 
student  learning  communities.  It  was  also  critical  for  us  in  structuring  the  learning 
experiences  for  the  faculty  learning  community. 

Although  a somewhat  different  aspect  regarding  student  learning  communities,  in  one  of  my 
graduate  courses  long  ago,  I began  the  course  by  initiating  the  students  into  a new  process, 
one  where  they  could  help  choose  the  course  content.  As  we  all  know,  we  choose  course 
content  based  on  what  we  know  and/or  identify  as  important  within  the  whole  body(ies)  of 
knowledge  across  the  field  or  related  fields.  So  there  is  a great  deal  of  choice  and  possibilities 
available  to  professors  regarding  disciplinary  knowledge,  skills,  and  information,  unless  one 
is  the  sort  to  choose  a text  and  merely  follow  the  text  (UGH!).  My  goal  was  to  engage  the 
students  in  more  of  a learning  community,  not  as  formal  as  the  one  described  above,  but 
where  learning  could  occur  using  a more  open  process  and  where  we  could  address  some 
critical  knowledge  that  was  desired  individually  and  collectively.  When  presenting  this 
opportunity  to  my  group  of  students  (a  typical  graduate  range  of  younger  to  older,  full  time 
working  engineers  and  managers  mixed  with  full  time  grads),  they  became  angry!  They  did 
not  want  the  choice;  they  wanted  me  to  dictate.  I refused,  and  we  got  through  it,  ending 
positively.  But  introducing  the  more  collaborative  approach  between  professor  and  student 
caused  a rather  tense  conflict  as  students  were  expecting  me  to  dictate.  That  was  the  model 
they  were  used  to  from  others,  and  they  initially  perceived  the  effort  as  too  demanding.  One 
or  two  questioned  me  about  whether  I was  doing  my  job.  Once  again,  this  is  a situation 
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where  you  may  know  that  you  are  using  the  best  teaching  and  learning  strategy,  so  you  must 
follow  through  even  though  your  end  of  course  evaluations  may  be  lower  initially.  Once  the 
process  was  introduced,  students  came  in  with  topics  in  hand  because  they  needed  it  for  work 
or  had  read  about  it.  The  point,  again,  is  that  the  professor  must  be  willing  to  try  different 
strategies,  models,  and  procedures,  fully  understanding  that  at  least  once  their  evaluations 
may  change,  not  much  but  somewhat  lower.  That  process  resulted  in  a modified  version.  We 
arrived  at  a combined  strategy  of  my  choices  and  theirs  and  created  more  of  a learning 
community,  where  there  was  an  open  forum  and  we  were  learning  more,  learning  from  each 
other,  and  the  content  was  extremely  relevant.  In  fact,  I realized  that  more  knowledge  was 
being  covered,  more  interactively.  With  experienced  graduate  students,  there  is  a wealth  of 
experience,  new  information,  and  validation  to  draw  upon.  Beside  other  validation  learning 
strategies  within  the  course  learning  processes,  the  professors  can  validate  content  as  relevant 
from  the  communities  of  practice,  business,  and  industry  for  the  inexperienced  or  younger 
students. 

When  Guteck  (as  cited  in  Andre  and  Frost,  1996)  discusses  learning,  she  writes  of  the  same 
question  that  has  often  come  to  me  about  course  content  and  process  "Why  are  you  changing 
your  courses  so  often?"  or  from  the  outside  world  "How  can  you  stand  to  teach  the  same 
courses  over  again?"  My  response  was  almost  identical  to  hers:  "They  are  not  the  same; 
there  is  new  content,  different  methods,  and  new  issues  each  time  I teach  them.  They  have  to 
change!  It  is  not  boring”  - but  the  other  part  of  my  answer  that  I do  not  always  express  is  that 
I have  learned  so  much  during  and  between  each  time  of  teaching  each  course.  My  research, 
travel,  reading,  conferences,  colleagues,  and  very  often  my  students,  especially  those  who 
work,  teach  me  so  much  in  so  many  ways  and/or  through  their  questions  or  presentation  of 
problems  stimulate  me  to  study  something.  It  may  not  be  always  new  knowledge,  but  the 
context  within  which  it  is  applied  or  employed  to  solve  a problem  becomes  modified.  One's 
existing  knowledge  is  "a  gift  that  keeps  on  giving,"  in  that  each  time  it  is  used,  questioned,  or 
presented  in  a different  context  that  knowledge,  information,  or  skill  extends,  deepens,  or 
becomes  useful  in  a different  context,  becoming  greater  and  different  than  it  was  and 
evolving  into  new  knowledge.  IF  we  are  learning  individuals  and  not  merely  the  "learned," 
there  is  hope  for  academics  to  build  exemplary  learning  communities  - those  that  should  be 
an  especially  desired  goal  are  the  faculty /student  learning  communities. 

However,  this  is  not  the  greater  majority  of  our  learning  scenarios.  Because  we  want  to  lead 
more  faculty  toward  student  learning  communities  and  circles,  and  ultimately  to 
faculty /student  learning  communities,  we  are  studying  the  work  that  has  been  done  to  date. 
Anderson  et  al.  (2003)  led  an  initiative  at  North  Carolina  State  University  with  other  research 
universities  to  develop  student  learning  communities  and  faculty  pedagogy.  For  us,  this 
means  that  there  was  a faculty  learning  community  that  engaged  in  learning  about  pedagogy, 
one  of  which  is  collaborative  learning  through  student  LCs,  and  then  in  the  design  and 
development  of  student  learning  communities,  not  always  through  intact  classes.  As  with 
FIGS  (freshman  interest  groups),  a higher  education  interdisciplinary  theme-based  learning 
culture  across  courses  and  professors,  learning  communities  can  be  structured  as  class  based 
for  students,  but  that  is  not  a requirement.  Faculty  learning  communities  can  be 
interdisciplinary  as  well.  Those  working  together  at  N.C.  State  determined  four  broad 
learning  outcomes  for  students:  (1)  critical  thinking;  (2)  habits  of  independent  inquiry;  (3) 
responsibility  for  one's  own  learning  and  intellectual  growth  and  development;  and  (4) 
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student  learning,  promoted  through  the  active  investigation  of  complex  questions  and 
problems  (p.  27).  These  seemed  to  be  appropriate  for  our  faculty  learning  community  as  well 
and  then,  in  turn,  for  their  consideration  of  student  learning  communities.  Anderson  et  al. 
note  that  LCs  are  widespread  because  they  can  be  used  or  adapted  flexibly,  "a  vision  of 
faculty  and  students  - and  sometimes  administrators,  staff,  and  the  larger  community  - 
working  collaboratively  toward  shared  academic  goals  in  environments  in  which  competition 
is  de-emphasized.... faculty  and  students  alike  have  both  opportunity  and  responsibility  to 
learn  from  and  help  each  other"  (Anderson  et  al.,  p.  1;  Angelo  1997,  p.  3).  They  worked 
together  to  restructure  the  curriculum  into  interdisciplinary  endeavors  around  themes, 
enrolling  a cohort  of  students,  similar  to  FIGS,  but  as  faculty/student  LCs. 

When  discussing  the  Faculty  Pedagogy  Workshops,  Anderson  et  al.  (2003)  stress  that  a 
common  language  is  needed  and  the  focus  should  be 

to  develop  classroom  practices  that  encourage  and  help  students  to  raise,  sharpen,  and 
follow  through  on  their  own  questions,  to  respond  to  questions  posed  by  the  faculty 
member  by  asking  further  questions  and  seeking  answers  to  them,  and  to  develop  a 
habitual  sense  of  inquiry  that  would  transcend  the  boundaries  of  the  course,  (p.  27) 

The  faculty  participated  in  symposia  across  four  professional  development  tracks:  1) 
Classroom  Teaching  Practices  and  Inquiry  Guided  Learning  (IGL);  2)  Technology  and  IGL; 
3)  Assessing  the  Effects  of  IGL  on  Students  and  Faculty;  and  4)  Other  undergraduate 
initiatives.  They  identified  their  changed  foci  from  three  sources:  "1)  what  our  teachers  told 
us  they  needed  and  wanted  to  know;  2)  assessment  of  our  programs;  and  3)  trends  and 
literature  on  faculty  development  and  best  practices  in  undergraduate  education"  (p.  34).  This 
also  reflects  our  strategy.  They  used  a summer  workshop  strategy;  we  are  including  time 
during  the  school  year  and  summer  as  well.  We  are  following  with  formal  research 
experimentation  in  the  classroom  in  our  move  towards  teaching  as  scholarship.  Anderson  et 
al.  (2003)  establish  that  student  learning  communities  are  most  common  in  research 
universities  but  make  no  judgment  about  more  comprehensive  universities.  A long  standing 
and  traditional  example  of  student  learning  communities  is  the  law  school  model  where 
students  form  and  belong  to  study  groups.  The  peer  support,  reinforcement,  and  methods  are 
used  to  distribute  responsibilities  individually  for  group  study  and  learning  continues  to  be 
practiced  and  work  well  for  students. 

Although  not  a student  group,  my  own  doctoral  dissertation  situation  somewhat  reflects  this 
type  of  opportunity  in  which  a faculty  committee  and  I engaged  as  a learning 
circle/community.  At  Ohio  State  University,  I took  a high  level  graduate  elective  in  reading 
comprehension,  and  in  the  process  met  one  of  the  most  outstanding  researchers  in  the  multi- 
field  of  reading,  comprehension,  information  processing,  etc.  Because  of  his  teaching 
strategies  and  student  learning  processes,  I was  motivated  to  want  to  learn  more  from  him 
and  was  stimulated  to  involve  myself  in  learning  at  a deeper  level  with  an  interdisciplinary 
focus.  My  field  immersed  me  in  technical  knowledge;  his  immersed  me  in  the  fields 
fundamental  to  learning,  those  of  processing  and  retaining  information.  This  led  me  to  decide 
that  I wanted  to  focus  my  dissertation  on  student  learning  and  technical  information 
processing.  Coming  from  a technical  field,  I was  not  prepared  for  such  research,  and 
although  my  committee  members  were  certainly  experts  about  learning,  they  were  not  about 
information  processing,  memory,  etc.  When  I discussed  this  with  my  advisor,  he  said  that  the 
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committee  did  not  have  the  eomplete  knowledge  set  to  direet  sueh  a dissertation,  BUT!  if  I 
eould  get  the  edueation  professor  to  be  the  direetor,  my  eommittee  members  would  be 
willing  to  learn  with  me.  This  was  rather  unusual. 

In  following  through,  I went  to  the  edueation  professor;  he  seratehed  his  head  and  said  to  me, 
“Do  you  realize  how  many  fields  of  literature  you  will  have  to  immerse  yourself  in  to  even 
begin  to  design  a simple  study  in  the  information  proeessing  of  knowledge?”  I said,  “No.”  He 
informed  me  I would  have  to  read  material  on  artifieial  intelligenee,  linguisties,  information 
proeessing,  learning,  and  more  - at  least  of  year  of  reading.  I was  truly  interested  and 
respeeted  him  so  highly  - not  to  mention  that  his  method  of  teaehing  had  ereated  sueh  a 
spark  or  stimulus  and  motivated  me  to  want  to  know  more  - he  agreed  to  try.  At  the  time, 
however,  I did  not  realize  that  approaehing  a dissertation  and  eommittee  relationship  like  I 
did  was  taking  a great  risk  and  eould  have  been  an  edueational  disaster.  But  beeause  of  the 
integrity  of  these  individuals  and  their  keen  desire  to  learn  as  well  - inherently  exemplars  of 
what  we  have  diseussed  above,  there  was  no  risk.  They  were  truly  exeited.  The  lead  professor 
in  edueation  took  on  a doetoral  student  he  really  did  not  have  time  for,  as  he  had  a high 
number  of  his  own.  And  my  eommittee  had  to  spend  more  time  learning  alongside  of  me  so 
they  eould  make  good  judgments  about  my  work  and  its  potential  eontribution  to  the  field  as 
researeh.  Together  my  teehnology  eommittee  members  and  this  edueation  professor,  all  of 
whom  were  only  willing  to  do  this  together  beeause  of  their  respeet  for  eaeh  other  as 
researehers,  teaehers,  and  most  importantly  learners,  worked  with  me  to  learn  enough  to 
design  and  exeeute  an  experimental  study  on  information  proeessing  of  teehnieal 
information.  This  is  truly  an  example  of  what  we  are  diseussing  here. 

What  a eommittee!  What  "speeial"  professors  who  spent  all  the  extra  time  it  took  for  them  to 
learn  with  me  sinee  they  eould  not  approve  what  they  did  not  know  about.  We  all  learned 
together.  It  was  terribly  diffieult,  although  the  study  may  not  refleet  how  diffieult  preparing 
to  do  it  was.  It  was  also  wonderfully  exeiting  and  stimulating!  It  took  me  where  my  natural 
interests  wanted  to  go  - aeross  diseiplines.  It  engaged  me  in  learning  well  above  my  eurrent 
knowledge  and  experienee;  important  for  their  outeomes  was  that  the  researeh  eontributed  to 
the  literature  and  body  of  knowledge  on  learning.  Most  importantly,  they  "let"  me.  This  was 
teaehing  and  researeh  entwined  at  its  best  from  both  the  student  and  professors'  perspeetives. 
To  me,  this  is  the  goal  for  us  in  higher  edueation.  Every  student  should  have  some  of  these 
stories  to  tell  about  their  edueational  experienees. 

We  will  gradually  move  our  faeulty  members  into  eonsideration  of  this  type  of  endeavor,  but 
our  initial  foeus  is  a faeulty  learning  eommunity  made  of  interdiseiplinary  learning  eireles, 
with  the  primary  learning  agenda  ineluding  teaehing  pedagogy,  student  learning,  student 
assessment,  and  edueational  researeh  as  we  entwine  teaching  as  scholarship  with  new 
teaehing  praetiees  and  a goal  of  inereased  student  learning.  Student  learning  eommunities 
and/or  faeulty-student  learning  eommunities  are  a teaehing  strategy  within  our  initiative 
eontext.  Anderson  et  al.  (2003)  inform  us  that  their  eontext  of  faeulty  development  as  a 
researeh  university  positions  them  to  deal  with  the  publie  eall  for  more  full  time,  senior,  and 
experieneed  faeulty  members  to  teaeh  undergraduate  students.  Their  model  is  truly  foeused 
on  the  interdiseiplinary  faeulty-student  model.  As  they  deseribe,  our  learning  eommunity  will 
fit  within  our  existing  strueture  and  will  be  eoordinated  by  one  individual.  Their  model 
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exemplifies  an  interdiseiplinary  faculty  learning  community  or  an  interdisciplinary 
faculty /student  learning  community  model. 

Shapiro  and  Levine  (1999)  provide  a practical  guide  for  creating  faculty/student  learning 
communities.  They  also  focus  on  the  development  of  faculty /student  and  student  LCs  and 
provide  a full  guide  on  the  issues,  strategies,  background,  etc.  very  similar  to  the  briefer 
version  by  Anderson.  This  will  be  an  important  resource  for  our  faculty  members  when  we 
determine  the  development  of  faculty-student  learning  communities  is  our  goal  or  strategy. 
Presently,  we  are  developing  a faculty  learning  community;  however,  their  exhibit  on 
learning  principles  and  collaborative  action  can  serve  to  inform  us  about  student  learning  as 
well  as  classroom  research. 

Figure  A.4.3:  Learning  Principles  and  Collaborative  Action 

EXHIBIT  6.2.  LEARNING  PRINCIPLES  AND  COLLABORATIVE  ACTION. 

1 Ltijming  ts  funiJamentallv  -ihoui  mskiruj  ,jn<l  connections: 

bic'touically  through  neiirji  outworks,  rrifnl.illy  itmong  concepts,  irJeas,  and 
mcvininqs,  ana  experientially  ttiiuiiigti  iMtei.uili.m  between  the  ^md  and  the 
crwironment,  self  -ind  other,  generality  and  context,  delitieruiiun  and  iiclion. 

J li  arrtirtg  in  enfi.iiu  rd  by  lakinq  place  in  the  context  of  ■)  conif tellim.)  suuotion 
that  tnii.tni  es  rliallemjo  .iritl  opportunity,  stimulating  and  utili,ring  the  bram';. 
ability  to  conce|HlJ,^ll/e  gmCKIy  on<l  its  r.jpacily  and  need  for  contemplation  an/I 
reflection  upon  e.-'/ppiipMce? 

3 le.-tiTiing  Is  -Jn  .icnve  search  tor  meaning  by  the  learner— cnnstrurling 

knowledge  rather  than  passively  receiving  if  shaping  as  well  as  being  shaped 
by  e.xperiences. 

4.  Learning  is  developmental,  a cumulative  pnx  ess  Involvniy  the  whole  penson, 
reliilinq  pa.sl  and  present,  integrating  me  npw  witn  rtie  nlO,  Starting  from  but 
Irarvscending  perscmnl  concerns  and  Interests. 

5.  Learninri  is  done  by  individuals  vvho  arc  intrinsicaify  tied  to  others  as  sori.al 
lifings,  irilerfic.liiiy  as  tompetilurs  or  callaboratrirs.  constmining  or  supoorlinu 
the  leainiiuj)  prcKess,  ,ind  able  to  enhance  learning  throuc^h  ccjope.Mtlc>n  .mij 
sh.inng 

6.  Leainirtg  is  strongly  aflecled  by  the  educational  climate  in  svhich  It  i.ikes  plac*-- 
dit-  settings  and  surroundiricj.s.  Die  influences  of  others,  and  the  values  firr/irrli  rf 
TO  tlir  life  of  ffie  iiUMd  and  to  Icorrijng  achievcmtnis. 

/ l»-,sii*ing  lequirt'S  freguenl  le»M.ib.ick  il  it  is  to  be  sustained,  piarlice  if  it  's  to  be 
numlstied,  ,ind  opportunities  lo  use  what  has  been  learned 

8 Miicli  li'.irning  tiikirs  place  inlurmally  and  incidentally,  beyond  explica  teaching 
ni  Die  (las'aooni,  in  casual  contacts  with  faculty  and  statf,  peers,  campus  life, 
at  live  social  and  Community  ins'nlvements,  and  unplannen  lint  ^prt'hy  ,itiil 
tiimplex  Miuaiions 

d I tviniing  is  grounded  in  particuUir  contexts  and  individual  expenen»  es, 

rf'c^uitiny  ('finri  to  translcr  specific  knowledge  and  skills  to  other  citcunistaiices 
or  to  more  general  understfindirujs  and  to  unlearn  pcftsonal  views  and 
approaches  when  confronted  by  new  information 
in.  Learning  involves  the  ability  of  inrjividuals  lo  monitor  th»‘lr  rjwn  learning,  to 
uneferstand  how  knowledge  is  acquired,  lo  develop  slratoyies  lor  learning 
ba.sed  on  discerning  their  capacities  and  limitations,  antf  to  be  aware  ot  their 
own  vVays  of  knowing  in  approaching  new  bodies  ot  knowledge  and 
disciplinary  Irtimeworks. 

i.iL//r  *>  Atnfcricaii  A.ssotiaiion  lor  Higher  Eduration,  American  College  l'ersonii*l  Associdiion,  .inci 
Nvitic'iMl  ^-,‘.Di.i/«ti"'i '•I  Slutiful  r*Tsnnnel  Administrators,  t fyH 

(Shapiro  & Levine,  1999,  p.ll9) 
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Lenning  and  Ebbers  (1999)  provide  a full  resource  for  developing  student  learning 
communities.  They  identify  the  types:  curricular,  classroom,  residential,  and  student-type 
learning  communities  and  itemize  the  benefits.  They  recommend  how  to  create  and 
implement  optimal  college  student  learning  communities  by  making  the  active  learning 
oriented  to  where  cooperative  or  collaborative  learning  is  a teaching  and  learning  strategy 
and  identify  the  problems  or  pitfalls  and  responses  or  potential  solutions  to  some  of  the 
problems. 

The  following  figures  from  Lewis  and  Allan  (2005)  provide  a model  for  virtual  student 
learning  communities  that  illustrate  the  dynamics  within  the  virtual  learning  community  and 
the  transfer  of  knowledge  between  members  of  the  community  (students  and  facilitator). 
Further  discussion  of  Lewis  and  Allan’s  work  can  be  found  in  the  literature  review. 


Figure  A.4.4:  Sample  Virtual  Learning  Community 


l 2. 1 virtual  k-ai  niiin  rommimitv 


(Lewis  & Allan,  2005,  p.  22) 


Figure  A.4.5:  Transfer  of  Knowledge 


Knowledge  and 
skills  transfer 
between  the  VLC 
and  workplace 


< ► 


Community 
members  including 
'facilitator' 


WORKPLACE 


WORKPLACE 


hpiTf  2.2  ft.m.sln  t»l  kium)(Mli;r  mlit  llic  VMukpiAi  t m a siinplt*  virtual  leaniimt 
comimmit' 


(Lewis  & Allan,  2005,  p.  24) 
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Figure  A.4.6:  Organizational  structure 


KrK)wlec29e  boo 
3ktlls  transfer 
DelA&on  the  VUC 
ana  workplace 


Members  come  from  a number 
of  organizations 


Facilitator  may  be  employed  by  one  of 
the  organizations  or  may  bo  a 
self-employed  conauttant 


hl^irr  2 • j ttmiiltr)  >«t  i»i>';ini/.^iions 

(Lewis  & Allan,  2005,  p.  25) 

Figure  A.4.7:  Series  of  Practitioner  Learning  Communities 


Senes  of  practitioner  learning  communities.  Each  one  rias  an 
internal  facilitator  and  tocusea  on  a panicular  topic,  problem  or  issue 

/tj.-  .'-  2 ^ CompliA  MiriuMr4nmit;tiimmiinm 


(Lewis  & Allan,  2005,  p.  28) 

These  authors  and  their  models  both  inform  and  confirm  the  direction  of  this  initiative  and 
the  strategies  for  accomplishing  its  mission. 
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REFLECTIVE  PRACTICE:  THE  SCHOLARSHIP  OF  TEACHING 
THE  CEET  FACULTY  DEVELOPMENT  MODEL  AND  PROGRAM 
Jule  Dee  Scarborough,  Ph.D. 


The  CEET  Initiative  on  Teaching  and  Eeaming  (CITE)  - the  Scholarship  of  Teaching  - began 
with  the  Dean  and  Eaculty  Eeader,  a distinguished  professor  and  teaching  and  learning  expert, 
meeting  to  develop  a college  vision,  mission,  goals,  and  faculty  development  program  focus.  The 
Dean’s  leadership  set  the  stage  for  the  program,  as  he  wanted  the  Scholarship  of  Teaching  to 
become  an  equal  realm  of  scholarship  for  professors  across  the  three  engineering  and  technology 
departments.  His  interests  extended  to  include  a desire  for  the  college  to  become  a regional  and, 
ultimately,  a national  leader  in  the  Scholarship  of  Teaching.  Included  in  the  overall  vision  was  to 
become  a leader  in  the  preparation  of  faculty  on  teaching  and  learning  so  they  could  better 
engage  and  become  leaders  in  the  Scholarship  of  Teaching.  Another  aspect  of  his  interest  was 
that  he  recognized  the  expertise  and  leadership  available  within  the  college  and  wanted  to  create 
an  environment  for  those  professors  recognized  outside  the  college  as  leaders  to  have  the 
opportunity  for  “local”  leadership  (internal  to  the  college)  with  their  peers;  thus  he  was  interested 
in  a Peer  Eeadership  Model.  Therefore,  his  interests  and  those  of  the  Eaculty  Eeader  in  the 
Scholarship  of  Teaching  at  the  university  level  meshed,  and  an  initiative  was  bom  for  the 
College  of  Engineering  and  Engineering  Technology  on  the  Scholarship  of  Teaching.  The 
Eaculty  Eeader  conceptualized,  developed,  and  led  the  program.  The  Dean  fully  funded  the  pilot 
initiative,  which  included  the  participation  of  seven  faculty  members  from  across  the  four 
departments,  the  program  leader,  one  program  associate,  a graduate  assistant,  and  resources  and 
materials.  The  budget  in  real  fiscal  commitment  was  over  $100,000,  fully  funded  by  the  Dean’s 
Office  (Boyer,  1990). 

The  NIU  CEET  Initiative  is  all  about  change.  To  implement  the  Scholarship  of  Teaching  through 
classroom  research,  there  was  a need  to  first  create  a faculty  learning  community  and  then 
provide  faculty  development  for  the  “community.”  A needs  assessment  was  performed,  overall 
goals  were  identified,  the  program  was  designed,  and  then  both  a literature  and  program  search 
were  conducted  to  determine  if  there  were  any  existing  programs,  in  part  or  whole,  that  matched 
our  needs.  We  found  no  programs  and  very  few  components  that  could  be  pulled  together  to 
create  the  program  that  we  felt  was  necessary  to  prepare  the  faculty  community  for  the 
Scholarship  of  Teaching.  The  underlying  philosophy  of  our  program  was  that  of  engaging  faculty 
members  in  a fully  integrated  program.  Teaching,  learning,  and  student  assessment  along  with 
several  other  topics  are  not  easily  separated,  as  they  are  truly  interdependent.  We  wanted  a 
“program,”  not  a series  of  workshops.  We  wanted  to  follow  our  faculty  members  into  their 
classrooms,  where  they  would  begin  their  experimental  research.  That  was  also  an  important 
aspect  of  the  “program”  approach  - to  include  the  classroom  research  as  a component  of  the 
faculty  development  program.  Once  the  needs  were  clearly  identified,  the  literature  was  reviewed 
and  documented,  and  finally,  the  university  Professional  and  Organization  Development 
Network  (POD),  a network  of  faculty  development  professionals  and  units  across  universities 
nationally,  was  tapped  to  see  if  any  other  university  had  programs  important  to  know  about  or 
draw  from.  We  then  developed  our  program  and  engaged  in  the  development  of  faculty  on  the 
Scholarship  of  Teaching. 
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Research 


There  were  two  research  projects  inherent  to  the  overall  Initiative.  (l)The  Faculty  Development 
Program  content  and  process  was  a pilot  development  and  research  project  and  (2)  the 
experimental  classroom  research,  executed  by  each  professor  on  teaching  and  learning  with  their 
students  during  the  research  semester  component  of  the  faculty  development  program,  was  the 
second  research  project.  Therefore,  the  CITL  - the  Scholarship  of  Teaching  - engaged  the 
college  in  two  substantial  research  projects  that,  although  integrated,  were  clearly  different  and 
defined  prongs  of  research  when  considering  the  Scholarship  of  Teaching:  (1)  the  faculty 
preparation  on  teaching,  learning,  and  educational  research  and  (2)  experimental  research  on 
teaching  and  learning  with  students  in  the  classrooms  (Boyer,  1990;  Campbell  & Stanley, 
1963,1966,  1977;  Zuber-Skerritt,  1992).  See  Final  Reports  on  both  in  the  A (Executive  Summary 
and  Table)  and  B (Detailed  Data  and  Report)  sections  of  this  document. 

Timeline 

The  actual  pilot  initiative,  not  including  the  preparation  and  planning  year,  began  with  a formal 
Invitational  and  Commitment  meeting  in  October  2005  and  ended  with  data  reports  in  January 
2007.  The  actual  faculty  development  program  began  in  February  2006,  ending  in  May  2006. 

The  research  semester  following  the  faculty  development  program  began  in  August  2006  and 
ended  in  December  2006.  Classroom  data  was  submitted  during  this  semester.  The  quantitative 
data  from  the  classroom  experiments  and  partial  qualitative  data  from  the  program  evaluation 
were  analyzed  and  presented  to  the  faculty  group  and  Dean  in  January  2007;  the  final  qualitative 
data  was  presented  to  the  faculty  and  Dean,  May  2007.  See  Faculty  Development  and  Professor 
Research  sections  of  this  document. 


Table  A.5.1:CIT] 

^ Summary  of  Timeline  (Basic  Timeline) 

Feb05  - Feb06 

Planning  and  Preparation 

Feb06  - May  06 

Faculty  Development  Program  (Data  Collection) 

Aug06  - Dec06 

Classroom  Research  Semester  (Data  Collection) 

Jan06 

Preliminary  Data  Analysis  -Critical  Data  Analysis 

Feb06-May07 

Completion  of  Data  Analysis;  Preparation  of  College  Portfolio 
Portfolio  Submitted  to  ERIC 

Jun06-Aug06 

NSE  Proposal  Preparation;  Eaculty  manuscripts  submitted  to  journals 
Portfolio  disseminated  to  engineering/technology  programs  nationally 
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'igure  A.5.1:  Reflective  Practice:  The  Scholarship  of  Teaching  - The  CEET  Model 
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The  above  model  represents  CEET’s  integrated  model  for  The  Scholarship  of  Teaching.  The 
model  integrates  faculty  preparation  and  classroom  research,  where  the  research  on  teaching  and 
learning  functions  was  the  culminating  learning  activity  within  the  faculty  development  program. 
The  model  is  inherently  designed  to  account  for  the  quality  of  the  program  and  the  ongoing 
change  process.  Thus,  the  CEET  model  incorporates  what  others  have  done  but  modifies  it  to 
become  a more  integrated  and  intentional  approach  to  critical  reflection  and  the  Scholarship  of 
Teaching  (Scarborough,  2006;  based  upon  Doming,  1950;  Dick  & Carey,  1996). 

The  CEET  model  incorporates  the  “Reversed”  Instructional  Design  Process  first  introduced  by 
Dick  and  Carey  as  Systematic  Design  of  Instruction  (1996),  where  professors  identify  what  they 
want  students  to  know  about  or  be  able  to  do  and  then  determine  what  assessment  will  best 
provide  evidence  that  the  learning  has  occurred.  Erom  that  point,  other  instructional  decisions  are 
made  (e.g.,  curriculum  context,  learning  activities,  teaching  models,  styles,  and  more).  Then 
Wiggins  & McTighe  (2005)  introduced  the  same  principles  as  “Backward  Design”  . Dick  and 
Carey’s  and  Wiggins  & McTighe’s  models  have  been  modified  for  our  purposes,  and  we  show 
the  entire  process  for  the  entire  initiative.  However,  a major  concept  underlying  our  efforts  can 
be  best  defined  by  the  term  “Intentional”  Instructional  Design;  we  adapted  “reversed”,  although 
long  a practice  in  technical  fields.  “Intentional”  may  best  be  explained  when  considering 
“natural”  teachers  or  those  professors  who  can  enter  a classroom  without  much  preparation  or 
thought  about  what  they  want  to  occur  and  something  good  will  happen;  students  will  learn 
something  important.  But  did  they  learn  what  they  should  have  learned?  Was  there  a planned 
focus  for  learning?  Instruction  should  be  “intentional”;  in  other  words,  the  “what-outcomes,” 
and  “assessment”  (evidence  of  learning)  should  be  thought  about,  identified,  and  prepared  for  in 
advance  of  walking  into  the  classroom.  When  a professor  leaves  the  classroom,  he/she  should 
leave  with  an  understanding  that  what  was  planned  or  intended  as  outcomes  for  that  day  occurred 
and,  at  some  point,  collect  evidence  of  the  learning  that  took  place.  If  for  some  reason,  students 
did  not  achieve  what  was  “intended,”  then  the  feedback  process  would  indicate  that  something 
needs  to  occur  to  ensure  learning. 

The  CEET  model  also  incorporates  the  Deming  Cycle  (1950s)  as  the  quality  model  and  an 
inherent  feedback  loop  that  streams  ongoing  quality  information  into  that  cycle  for  continuous 
improvement.  The  model  reflects  a closed  loop  process  for  ongoing  change.  Thus  evaluation  is 
incorporated  more  naturally.  CEET’s  model  is  comprehensive  in  that  it  prepares  faculty  for 
teaching  and  learning  and  educational  research.  The  learning  process  engages  professors  in  the 
redevelopment  of  one  of  their  own  courses  as  the  vehicle  to  learn  about  teaching  and  learning 
and  then  adds  a component  on  educational,  experimental  classroom  research  to  prepare  them  for 
the  research  (and  evaluation)  aspect,  using  both  the  redeveloped  course  and  classroom  research 
as  the  vehicle  for  the  professors  to  engage  in  their  first  and  formal  experience  on  the  Scholarship 
of  Teaching. 

The  process  for  faculty  development  was  often  very  simple,  while  at  other  times,  fairly  complex. 
Although  there  were  presentations,  each  noted  on  the  program  description  and  calendar  below, 
the  instructional  model  for  the  program  was  outcomes  oriented,  performance  based,  learner- 
knowledge-  and  assessment-centered,  and  also  very  actively  engaging.  The  professors  were 
assessed  for  knowledge  and  skill  gains  throughout  the  entire  program.  The  program  was 
somewhat  collaborative;  in  this,  I mean  that  faculty  members  were  very  focused  on  the  analyses 
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and  redevelopment  of  their  own  individual  courses  and  instructional  decisions,  so  they  did  not 
“engage”  together  to  perform  most  of  the  development.  However,  they  were  encouraged  to 
engage  with  each  other  for  ideas,  examples,  assistance,  or  discussion  at  any  time.  Also  there 
were  formal  collaborative  group  forums,  discussions,  and  exchanges  many  times  throughout  the 
program.  There  was  also  formal  cooperative  learning,  where  faculty  members  engaged  in  the 
jigsaw  process  to  bring  different  information  on  a topic  together  to  complete  the  whole  picture  or 
perspective  from  a variety  of  sources  individually  assigned.  This  made  learning  more  efficient  as 
well  as  engaged  the  faculty  members  in  formal  cooperative  or  collaborative  learning.^  The 
professors’  (student)  learning  outcomes  for  the  program  were 

I.  To  analyze  each  existing  course  to 

a.  determine  appropriate  content  knowledge  for  achieving 
ABET(EAC/TAC)/NAIT  standards  or  student  learning  outcomes 

b.  determine  knowledge  content  priority:  major,  secondary,  other  or  minor 

c.  determine  how  knowledge  fits  into  Bloom’s  Taxonomy  Knowledge 
dimensions 

d.  determine  the  embedded  general  education  goals 

e.  determine  appropriate  teaching  models  and  styles 

f.  determine  which  student  learning  styles  are  being  engaged 

g.  determine  the  levels  of  Bloom’s  Taxonomy  Dimension  of  Eeaming  are 
being  achieved 

h.  determine  the  levels  of  Dale’s  Cone  of  Eearning  being  achieved  passive  - 
active 

i.  determine  strengths  and  weaknesses  of  the  course 

j.  determine  strengths  and  weaknesses  of  instruction 

k.  determine  strengths  and  weaknesses  of  syllabus 

l.  determine  strengths  and  weaknesses  of  student  learning 

II.  To  analyze  all  tests  to 

a.  determine  the  overall  quality  of  the  test 

b.  determine  the  overall  quality  of  test  items 

c.  identify  strengths  and  weaknesses  of  existing  tests 

d.  map  test  relationship  to  course  outcomes 

e.  map  test  items  to  course  outcomes 

f.  analyze  other  assessments  (very  few)  for  quality 

g.  analyze  other  assessments  (very  few)  relevant  to  learner  outcome 

III.  To  redevelop  course  outcomes  that  directly  link  to  ABET(EAC/TAC)/NAIT 

a.  redevelop  the  course  outcomes  and  map  relationship  to 
ABET(EAC/TAC)/NAIT 

b.  break  down  outcomes  in  outline  form  - major,  secondary,  minor  levels 

c.  identify  knowledge  according  to  Bloom’s  Knowledge  Dimensions 

d.  identify  embedded  general  education  goals 

e.  map  outcomes  to  Bloom’s  Dimension  of  Eearning  levels 

f.  map  outcomes  to  Dale’s  Cone  of  Eearning  levels 


* All  outcomes  achieved  level. 
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IV.  To  re-develop  tests  that  directly  link  to  course  outcomes  and 

ABET(EAC/TAC)/NAIT 

a.  create  a table  of  specifications 

b.  develop  a bank  of  diverse  test  items,  multiple  items  for  each  outcome 

1 . multiple  choice 

2.  true/false 

3.  short  answer 

4.  matching 

5.  problems 

c.  assemble  two  comprehensive  tests 

1.  midterm 

2.  final  examination 

d.  administer  newly  developed  tests 

V.  To  develop  a more  multifaceted  and  balanced  student  assessment  plan 

a.  develop  three  complex  performance  tasks  with  corresponding  rubrics 

1 . task  and  rubric  that  corresponds  with  the  midterm  exam 

2.  task  and  rubric  that  corresponds  with  the  final  exam 

3.  task  and  rubric  to  further  enhance  the  more  balanced  assessment  plan 

4.  incorporate  student  self-assessment  using  rubrics 

b.  develop  other  types  of  student  assessments  to  further  diversify  and 
balance  the  course  assessment  plan;  choose  from  or  determine: 

a.  quizzes 

b. 

c.  projects 

d.  case  studies 

e.  papers 

f.  reports 

g.  literature  reviews 

h.  design  problems 

i.  presentations 

j.  concept  mapping 

k.  team  projects 

l.  field  experiences 

m.  simulations 

n.  portfolios 

c.  employ  student  self-assessment  procedures  on  particular  or  all  assessments 

VI.  To  reconsider  grades,  grading  criteria  and  processes 

a.  eliminate  curving  of  grades 

b.  determine  grading  criteria 

c.  determine  scoring  protocols 

d.  implement  rubrics 

e.  implement  student  self-assessment 

f.  determine  formal  course  assessment  grading,  scoring  structure 
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VII.  To  reconsider  other  instructional  decisions  by  increasing  the  repertoire  of  options: 

a.  choose  a broader  repertoire  of  teaching  models  to  use  in  the  redeveloped 
course 

b.  choose  a broader  repertoire  of  teaching  styles  to  use  while  teaching  the 
re-developed  course 

c.  provide  a wider  range  of  learning  opportunities  that  engage  a more  diverse 
range  of  student  learning  styles 

d.  consider  multiculturalism  and  its  effect  on  student  learning  and  planning 
instruction 

e.  consider  student  motivational  factors  in  making  instructional  decisions 

f.  consider  student  perception  factors  in  making  instructional  decisions 

g.  consider  improvements  of  learning  environment  and  learning  space 
arrangements  (possibly  second  program  phase) 

VIII.  Determine,  design,  develop... finalize 

a.  contextual  curricula 

b.  learning  activities 

c.  group  or  team  learning  and  assessment  processes 

d.  other  instruction  delivery  (e.g.,  learning  style  inventory) 

IX.  Redesign  and  develop  new  course  syllabus  incorporating  the  following 

categories: 

a.  professor,  graduate  assistant  contact  information 

b.  catalog  course  description 

c.  course  purpose 

d.  course  requirements:  text,  datebook,  curricular  course  packets,  etc. 

e.  course  pre-  or  co-requisites 

f.  expected  computer  use,  knowledge,  skills,  software,  etc. 

g.  student  learning  outcomes,  identifying  embedded  general  education  goals,  and 
showing  connection  to  ABET(EAC/TAC)/NAIT  outcomes  with  links  to 
assessments 

h.  course  schedule/timeline  showing  course  weeks/days,  topics,  activities,  due 
dates,  lectures,  tests,  projects,  fieldtrips,  etc. 

i.  course  requirements:  list  assessments  and  points,  percentages,  structure,  etc. 

j . grading  structure 

k.  academic  misconduct  or  cheating  policy 

l.  professor’s  role;  graduate  assistant  role 

m.  professor’s  notes:  particular  notes  about  expected  behavior,  rules,  tardiness, 
absenteeism,  cell  phones,  late  assignments,  etc. 

n.  support  services  available  to  students,  e.g..  Writing  Center,  tutorial  services, 
accessibility /accommodations  services,  etc. 

o.  course  references 

p.  course  requirements  explanation  - description  of  each  type  of  assignment 

q.  course  requirements  check  off  - list  of  all  assignments,  projects,  activities 
with  point,  percentage,  scoring,  or  grading  information  so  students  can  keep 
track  of  their  progress  in  course  more  easily. 
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X.  Conduct  classroom  research  on  teaching  and  learning 

a.  design  research 

b.  select  methodology  and  procedures 

c.  conduct  experiment 

d.  collect  data 

e.  analyze  and  interpret  data 

f.  develop  conclusions  and  recommendations 

g.  prepare  manuscripts  for  publication 

The  learning  process  was  led  rather  than  “imparted.”  A myriad  of  worksheets,  information 
handouts,  and  tasks  were  used  to  accomplish  most  of  the  work.  Although  extremely  intense  and 
demanding,  the  analyses,  learning  processes,  and  development  activities  were  directly  tied  to 
educational  products  and  strategies  to  be  used  during  the  redeveloped  course  when  it  was  offered 
Fall  2006,  the  experimental  research  semester.  (See  Portfolio  Section  Cl)  Therein  is  the 
foundation  for  the  commitment  and  work  ethic  required  of  the  professors  and  also  demonstrated 
by  the  professors.  Everything  they  engaged  in  was  for  the  course  improvement,  enhancement  of 
their  teaching,  and  ultimately  increased  student  learning. 

To  measure  professor  learning  on  each  program  component,  both  objective  tests  and 
performance  tasks  were  used  as  well  as  alternative  assessments  (e.g.,  self-competency 
assessments  and  others).  Sometimes  multiple  tools  were  employed  to  measure  professor 
learning.  The  educational  products  or  tools  developed  by  the  professors  were  the  result  of  their 
performances;  these  were  judged  qualitatively  to  determine  the  quality  and  level  of  their 
achievement.  Thus,  the  faculty  development  program  involved  assessment  for,  of,  and  as 
learning  using  traditional,  performance,  and  alternative  assessments.  For  a full  discussion  on 
assessment,  especially  performance  assessment,  see  Scarborough,  2004,  Chapter  12,  copied  into 
this  document.  See  Portfolio  Sections  A5. 

Program  Description 

The  results  of  this  program  can  be  found  in  Section  B.0-B.13. 

Planning  and  Preparation  for  the  Initiative  (February  2005  - February  2006) 

The  program  was  based  upon  relevant  past  and  current  research,  studies,  best  accepted  practices 
and  models  in  the  literature,  and  books  by  a host  of  well  known  and  nationally  acclaimed  authors 
on  teaching,  learning,  assessment,  educational  research,  and  more.  The  program  leader 
conceptualized  a program  based  upon  experience  working  with  grant  funded  teaching  and 
learning  initiatives  with  high  schools  and  partner  post- secondary  education  institutions.  The 
work  of  these  partnerships  was  complex;  involved  large  numbers  of  teachers,  instructors,  and 
professors;  and  provided  extended  time  with  the  groups  (Scarborough,  2004; 
strategicalliance.niu.edu).  The  national  call  for  the  Scholarship  of  Teaching  by  Boyer  (1990)  and 
The  Carnegie  Foundation  motivated  the  design  and  development  of  a “program”  for  higher 
education,  and  although  it  can  be  used  for  any  group  of  professors,  the  program  leader  was 
predominantly  concerned  with  engineering  and  technology  professors.  To  confirm  the  ideas  for 
the  program  and  to  more  fully  validate  the  program  concept  and  topics,  a full  literature  review 
focusing  on  teaching  and  learning  at  the  university/college  level  was  prepared.  Rather  than  have 
each  professor  perform  his/her  own  complete  review,  it  was  determined  that  one  effort  on  the 
part  of  everyone  involved  would  be  more  efficient,  productive,  and  cost  effective.  Therefore,  all 


8 


participating  faculty  members  were  provided  the  entire  review,  with  a great  many  hardcopies  of 
selected  articles,  books  on  critical  topics,  and  examples  of  educational  products  available  from 
the  literature  (e.g.,  different  types  of  rubrics,  etc.).  The  literature  search  and  review  was  wider 
and  deeper  to  not  only  validate  the  program  content  but  also  to  provide  direct  program  support 
for  the  professors.  Since  the  program  was  comprehensive,  the  review  had  to  reflect  that 
framework.  It  covered  an  extensive  realm  of  literature  on  teaching  and  learning;  most  of  it  can  be 
viewed  within  this  portfolio.  The  literature  review  purpose  extends  beyond  the  traditional 
purpose  of  establishing  the  baseline  and  justification  for  research  and  development.  Our  review 
is  also  for  the  purpose  of  teaching.  We  wanted  a review  that  would  make  it  easier  for  the 
professors  to  begin  to  develop  their  own  schema  about  teaching  and  learning.  Although  it  would 
have  been  a great  foundation  for  them  to  each  study  more  deeply  the  wide  range  of  literature 
available  on  teaching  and  learning,  we  had  to  consider  motivation  and  time  and  decided  to  try 
introducing  them  to  the  literature  this  way.  Therefore,  a complete  list  of  articles,  books,  etc.  is 
included  that  reveals  the  information  in  each  professor’s  educational  toolbox.  There  is  also  a 
third  reference  list  more  directly  related  to  engineering.  The  program  leader  began  to  prepare  for 
the  initiative  approximately  one  year  before  the  work  with  the  professors  began.  That  also  gave 
the  Dean  time  to  allocate  the  budget  needed  for  initiative  execution.  See  References,  Toolbox 
Lists,  and  Sections  A8. 

Participants  and  the  Commitment  Meeting  (October  2006) 

Once  the  preparation  aspect  was  somewhat  completed,  the  selected  professors  were  invited  to 
participate  by  the  Dean.  In  determining  who  would  be  invited  for  the  pilot  initiative,  we  decided 
that  a diverse  group  was  best  and  that  it  should  include  individuals  who  would  be  both  interested 
in  such  an  initiative  and  willing  to  engage  sincerely  to  accomplish  the  goals  of  the  faculty 
development  program  and  research  semester.  Although  not  randomly  chosen,  as  preferred  for  the 
research  aspect,  the  professors  ranged  widely  in  their  teaching  capability,  styles,  methods,  and 
experience.  Clearly,  however,  all  were  engineers  and  technologists  with  no  formal  background  in 
teaching,  learning,  assessment,  or  educational  research.  Ultimately,  the  purpose  of  the  meeting 
was  to  (a)  explain  the  full  realm  of  the  initiative;  (b)  help  each  potential  participant  realize  the 
level  of  involvement  that  would  be  required;  (c)  explain  the  initiative’s  relationship  to  the 
professoriate,  professors’  responsibilities  and  duties;  (d)  explain  the  reward  structure;  (e) 
answer  all  questions;  and  (f)  confirm  the  commitment  of  each  faculty  member.  Table  1 below 
provides  the  program  calendar.  Each  program  component  is  explained  in  the  following  sections. 
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Table  A.5.2:  CEET  Initiative  on  Teaching  & Learning  Spring  Semester  Schedule  (2006)  (*assessment) 


Thursday,  Feb.  2 

Thursday,  Feb.  9 

Thursday,  Feb.  16 

Thursday,  March  2 

Thursday,  March  23 

Orientation  - Presentation 

Course  Anaivsis  - Presentation 

Course  Anaivsis  (cont.) 

Course  Anaivsis  (cont.) 

Course  Development 

Facnlty  Roles,  Respons.,  Duty 

Knowledge  Content  Outlines 

Objectives  & Outcomes  matched  to 

Teaching  Models 

ABET/TAC/NAIT  Standards 

The  Scholarship  of  Teaching  - The  National 

Knowledge  Priorities 

assessments 

Teaching  Styles 

- Student 

Call  for  Action 

Embedded  General 

Learning  Styles 

Outcomes 

Action  Research 

Education  Goals 

Assessments  by  Bloom  and  Dale 

Instructional  Design 

Outcomes  by  Bloom’s 

Learning  Communities 

Student  Learning 

Critical  Thinking 

Analysis 

Knowledge  Communities 

Objectives  and  Outcomes 

Course  Calendar 

Communities  of  Practice 

Teacher,  Knowledge,  Assessment,  or 

Double  Loop  Learning 

“Reversed”  Instructional  Design 

Learner  Centered? 

Introduction 

Teaching  Professionals 

Model  - Intentional  Design 

Complete 

Syllabus  Development 

What  is  Learning? 

Test  Anaivsis  - Presentation 

GAPS  Analysis  Summary 

Super  Syllabus 

What  is  Learning  Pedagogy? 

Taxonomies  of  Learning 

Purpose  of  Test  Analysis 

Dale’s  Cone  of  Learning 

Item  Analysis 

Self-Assessment:  The  First  Step  in 

Item  Difficulty 

Active  Learning 

Reflective  Practice 

Objectives  & Outcomes 

Item  Discrimination 

Problem-based  Learning 

By  Bloom’s  Taxonomy  and  Dale’s 

Case  Test  Analysis 

Growing  up  Digital 

Program  Description  & Model 

Cone 

Elagged  Items 

Student  SWOTs  Analysis 

Analysis  of  Results 

Syllabus  Analysis 

Validity 

10/11/05-  Commitment  Mtg. 

Reliability 

-Program  Description 

Standard  Error  of 

-Requirements 

Measurement 

-Timeline 

Using  NlU’s  Testing  Services 

-Expected  Outcomes 

* Student  Learning  Objectives 

-Rewards 

*SLO  Assessment  (a) 

*Test  Analysis  Assessment  (a) 

-Outcomes  Assessment  - SLO 

*LC  Assessment  (a) 

(a) 

*Self-Competency  (a) 

Thursday,  March  30 

Thursday,  April  6 

Thursday,  April  20 

Thursday,  April  27 

Break  April  28-May  14 

Test  Development  - Presentation 

Test  Development 

Test  Development 

Test  Development 

See  you  in  May! 

Discrete,  Objective  Items 

Test  Items/Bloom’s  Taxonomy 

Item  Writing 

Item  Writing 

Midterm  Exam  Test  Assembly 

Case  Test 

Valid  Test  Items 

Final  Exam  Test  Assembly 

Constructing  Multiple  C.  Items 

Test  Development 

Constructing  Short  Answer  It. 

Develop  Items-suhmit 

*Test  Development  Reassessment  (b) 

*Test  Analysis  Re-assessment  (b) 

*Test  Dev.  Assessment  (a) 
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Table  A. 5. 3:  CEET  Initiative  on  Teaehing  & Eeaming  IV 

ay  15-25  Schedule  (*assessment) 

Mon.,  March,15-,  9am-5pm 

Tues.,  March  16,  9am-5pm 

Wed.,  March,  17,  9am-5pm 

Thurs.  March,  18  9am-5pm 

Fri.,  Mar.  19 

ResrouD  Dav  -Test  Analysis, 

Performance  Assessment  & 

Reflectiye  Practitioners  - Presentation 

Balanced  Assessment 

9am-5pm 

Develonment,  Assembly  Reyiew 

Rubrics  - Presentation* 

Deyelopment  PAR  #2 

Course  Assessment  Plan 

Teaching  Models 

SLO/Test  Item/Bloom  Analysis 

Performance  Assessment  & Rubric 

PAR  deyelopment  - finalize  Perform  PAR  Bloom 

ON  OWN 

(PAR) 

Share  and  Critique  with  Peers 

Analysis 

Continue  SLO  actiyities 

Deyelopment  PAR  #1 

Deyelopment  PAR  #3 

Other  Assessm.  types 

Reading  Assignment 

TEACHING  PORTFOLIO 

Map  Assessments  (Kuhs  et  al.) 

Portfolio  Organization 

Share  and  Critique  with  Peers 

Share  and  Critique  with  Peers 

Identify-  types  of  assessment  to  include  in  Course 

Learning 

THE  CITE  TOOL  BOX 

*See  PA  Chapter  & Ref.  Materials 

A.  Plan 

Activity 

Tool  Box  Organization 
Each  Professor  builds  his/her 

Deyeloo  descriptions,  products  for  each  type 

own  Tool  Box 

*See  PA  Chapter  & Reference  Materials 

*PAR  Assessment  - Products 

*TM  Assessm. 
Product 

*SLO  Reassessment  (b) 

Explain  Friday’s  Assignment 

*TD/A  Re-assessment  (c) 

Monday,  M22  9am-5pm 

Tuesday,  M23  9am-5pm 

Wednesday,  M24  9am-5pm 

Thursday,  M2S-9am-5pm 

Summer!! 

Cooneratiye  Learnins  - Presentation 

Multi-Culturalism  in  Course 

Grading 

9-11  Classroom  Research  Experiments  - 

See  you  on: 

Apply  MC  in  course 

What  Competencies  do  grades 

Presentation 

l.Summer- 

Mannina  as  Assessment  & 

communicate? 

Research  date 

Actiye  Learnins  - Man  Courses 

*Reyisit  Cooperatiye  Learning 

Plan  LC  Goals  and  Actiyities 

Reyiew  Teaching  & Learning 

Complete  Assessment  System 

Complete  Assessment  System 

12-1  Lunch  with  Dean  Vohra 

Scholarship  of  Teaching 

2.Fall  dates: 
a.  Regroup 

-Reyiew  TM/TS/CL/M  Mean/Use 

Complete  T/L  Decisions 

Program  Assessment  DISC. 

2:00-4:00  Dean  Vohra 

-Reyiew  St.  LS-Kolb  Model 

Dept.  Chairs  Meeting/Profs. 

h.  MT/Einal 

-Reyiew  Dale's  Cone-Analyze  A. 

What  does  your  Syllabus 

Complete  Portfolio 

Test  Analysis  & 

Course  Content  Schedule  Teaching 

COMMUNICATE  - or  NOT!! 

Program:  Presentation 

Reyiew  date 

Decisions  - TM,  TS, 

Teaching  & Learning  Assign. 

Reflectiye  Practice  & Change 

St.  LS 

The  MODEL  SYLLABUS 

Scholarship  of  Teach.  Assgn. 

Learning  Community 

c.  End  of  Semester 

Syllabus  Completion 

Teaching  & Learning 

Course  Calendar  Completion 

Grading  Assignment-JS 

Discuss  Dean/Chair  Meeting 

Scholarship  of  Teaching 

d.  Data  Reyiew 

Multicultural  Assignment  -JS 

*Portfolio  Assessment  (a) 

Set  Fall  LC  dates 

e.  Art.  Meet. 

* Mao  Proaram 

4-5pm  *Program  Assessment 

*LC  Reassessment  - (h) 

*Self-Competency  Re-ass(b) 
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Program  Orientation  (February  2006) 

The  program  orientation  included  an  introduction  to  the  Scholarship  of  Teaching  and  an 
explanation  about  learning  communities,  with  a focus  on  what  learning  means.  Orientation 
continued  with  a full  description  about  what  the  faculty  development  program  focus  and 
topics  would  be  with  a description  of  all  educational  products,  instructional  decisions,  and 
activities  expected  from  the  professors  as  well  as  a description  of  the  operational  model: 
assessment  as  learning;  active  and  engaged  learning;  and  traditional  and  performance  based 
assessment.  The  program  leaders  described  that  the  standard  for  teaching  the  program  would 
be  they  would  use  the  teaching  models  and  styles  of  focus  within  the  program  content  and 
they  would  also  model  best  practices  and  behaviors  when  leading  the  program.  Finally,  the 
learning  community  goal  was  discussed;  the  process  would  be  one  where  the  faculty  could 
engage  to  evolve  their  own  learning  community  as  an  interdisciplinary  group.  Orientation 
concluded  with  an  explanation  about  “Intentional”  Instructional  Design  and  the  “Reversed” 
Design  process  (Dick  & Carey,  1996).  Finally,  the  professors  engaged  in  a basic  SWOTs 
(Strengths,  Weaknesses,  Opportunities  to  Improve,  and  Threats  with  addition  of  “s”  for 
solutions  where  there  are  weaknesses  or  threats)  analysis  of  students  and  student  learning. 
Some  professors  viewed  this  from  a program  and  teaching  perspective;  others  from  the 
student  perspective;  and  still  others  from  both  perspectives.  Nevertheless,  the  results 
provided  another  type  of  thought  provoking  validation  for  the  faculty  development  program 
and  research  initiative.  See  Presentation  PowerPoint  - D2 

Faculty  Development  Program:  Comments  on  Teaching  and  Learning 
It  is  important  to  understand  that  these  professors,  even  those  who  had  attained  some 
knowledge  or  had  some  sense  of  the  educational  aspects  of  teaching  and  learning,  had 
minimal  to  no  background  on  the  art  and  science  of  teaching  and  student  learning.  A few  had 
attended  workshops  or  read  articles  and  books  and  then  tried  some  of  what  they  learned  in 
their  own  classes.  None  had  engaged  with  peers  on  teaching  and  learning.  Therefore,  when 
designing  the  program,  our  struggle  was  about  how  much  breadth  and  depth  to  provide  on  the 
knowledge  content  of  teaching,  student  learning,  assessment,  and  instructional  models, 
practices,  or  procedures,  and  educational  research  because  each  topic  could,  in  itself,  be  a 
course. 

Philosophically,  the  leader’s  goal  was  to  provide  a “program”  rather  than  a series  of 
artificially  separated  workshops  because  a program  presents  the  content  as  it  really  plays  out 
in  the  classroom  - integrated;  a program  provides  the  opportunity  for  participants  to  actively 
perform  and  produce  what  they  need  in  their  classroom.  A program  does  not  end  with 
presentation  of  information;  it  takes  the  professors  from  analysis  of  courses  and  instructional 
practices  through  learning,  making  changes,  and  ultimately,  performing  in  the  classroom.  We 
made  the  decision  to  go  broad  in  program  content  with  more  depth  and  time  on  particular 
topics.  We  wanted  the  parameters  to  include  course  analysis  to  identify  strengths  and  needed 
improvements,  the  determination  of  student  learning  outcomes;  student  assessment  beyond 
tests  (performance  assessment);  and  something  on  grading.  We  also  wanted  to  address 
teaching  models,  styles,  student  learning  styles,  and  multiculturalism,  student  motivation,  and 
perception  factors.  We  did  this  and  more  with  the  greatest  amount  of  time  spent  on  course 
analysis,  testing,  performance  assessment,  and  teaching  models.  See  the  Program  Calendar 
and  Portfolio. 
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Another  difference  in  a program  versus  a workshop  series  approach  should  be,  and  was  for 
us,  that  we  tried  to  provide  each  professor  with  a “toolbox”  of  the  references  and  materials 
needed  to  continue  on  their  own  after  the  faculty  development  and  classroom  research 
experience  were  completed.  We  have  found  through  previous  experiences  with  other  groups 
that  individuals  and  small  groups  will  continue  to  evolve,  especially  if  they  have  everything 
needed  right  in  their  own  offices.  Therefore,  each  of  our  professors  left  the  program  with  a 
“toolbox”  or  files  of  articles,  books,  examples,  worksheets,  aids,  and  more  to  make  it  easy  for 
them  to  continue  changing  as  teaching  professors  and  to  also  change  their  other  courses.  Also 
we  were  committed  to  modeling  the  models,  techniques,  and  behaviors  that  we  were  trying  to 
evolve  the  professors  into  formally  using  in  the  classroom. 


Figure  A. 5. 2:  CEET  Eaculty  Development  Program  Models  of  Teaching  Map 

CEET  Faculty  Development  Program  : Models  of  Teaching 


Having  explained  our  philosophy,  it  is  important  to  understand  that  in  18  full  8-hour  days, 
plus  several  other  shorter  meeting  days,  we  covered  a lot  of  topics  broadly,  and  some  more 
deeply.  The  program  was  learner-knowledge- and  assessment-centered  using  both  self-  and 
other  formal-assessment  procedures,  but  with  a wide  variety  of  assessment  opportunities, 
including  the  culminating  ones  of  a Teaching  Portfolio  followed  by  experimental  classroom 
research.  A brief  description  of  the  program  and  each  component  is  presented  below. 
Portfolio  products  are  identified  where  appropriate.  The  professor  assessment  results  are 
presented  in  the  Data  and  Reports  section.  (See  Portfolio  Sections  Bl-13) 
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Figure  A. 5. 3:  CEET  Eaculty  Development  Program  Assessment  Map 


In  striving  to  validate,  benchmark,  or  inform  ourselves  further,  the  search  for  faculty 
development  “programs”  sought  not  only  current  and  relevant  literature  but  also  offerings 
from  faculty  development  offices  across  universities  nationally.  The  Director  of  NIU’s 
Eaculty  Development  Office  sent  out  an  email  requesting  information  on  any  type  of  related 
endeavor,  whether  workshop,  seminar,  or  program.  There  were  four  responses;  the  content  of 
those  responses  are  described  in  the  Eaculty  Development  literature. 

It  might  be  important  to  know  that  we  considered  the  “burden  of  learning”  and  who  was 
shouldering  that  burden  in  the  professors’  classes.  In  analyzing  their  courses,  teaching 
models,  teaching  styles,  student  learning  styles,  and  the  level  of  Dale’s  Cone  and  Bloom’s 
Taxonomy  that  the  course  or  student  learning  was  achieving,  the  professors  began  to  realize 
that  the  “burden  of  learning”  was  on  them  as  professors,  instead  of  on  the  students  where  it 
should  be.  The  courses  were  teacher-and  knowledge-centered  rather  than  balanced  across 
learner,  knowledge,  and  teacher  assessment.  They  learned  that  their  courses  needed  to  be 
refocused  so  that  the  burden  of  learning  would  become  accepted  and  sought  after  by  their 
students,  thus  transferring  that  burden  from  themselves  to  their  students  and  motivating  the 
students  through  ownership  of  their  own  learning.  This  perspective  does  not  free  the 
professors  from  continuously  learning  of  new  or  deeper  knowledge  in  their  respective 
disciplines;  instead  it  expands  their  activities  to  include  true  leadership  of  the  learning 
process  in  their  classes,  where  they  lead  instead  of  “impart,”  where  students  are  active 
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participants  instead  of  passive,  and  where  inquiry  and  diseovery  are  the  foundation  for 
learning.  In  the  multiple  analysis  process,  the  assessments  were  analyzed  equally  beside  the 
student  learning  outeomes,  eourse  eontent,  teaehing  models,  styles,  and  student  learning 
styles.  Throughout  our  sessions  on  student  learning  assessments,  test  analysis  and 
development,  and  performanee  task  and  rubrie  development  were  developed  towards  higher 
levels  of  Bloom’s  Taxonomy  and  Dale’s  Cone  as  well.  Often  the  foeus  of  faculty 
development  is  on  improving  instruetional  praetices,  and  the  improvement  of  student 
assessments  is  not  ineluded.  CEET’s  program  addressed  both.  It  eonneeted  the  use  of 
assessment  analysis  as  a diagnostie  tool  for  improving  student  assessment  and  instruetion. 

A fundamental  underpinning  of  the  entire  program  was  the  use  of  Bloom’s  (1956;  2001) 
(original  and  revised)  Taxonomy  of  Beaming,  both  the  Knowledge  and  Cognitive  Proeess 
Dimensions  and  Dale’s  (1969)  Cone  of  Beaming.  Educators  have  long  argued  about  the  use 
of  these  models  in  planning  eurriculum,  teaehing,  and  learning  activities,  or  student 
assessment.  We  made  the  deeision  to  use  these  as  fundamental  parameters  for  almost 
everything  we  analyzed,  eonsidered  and  developed.  Einally,  it  is  important  to  know  that  other 
fundamental  operational  models  for  us  were  the  “Baekward”  Instructional  Design  (Wiggins 
& MeTighe,  2005)  and  Systematic  Instructional  Design  by  Diek  and  Carey  (1996).  We  used 
the  basie  models  and  then  modified  them  to  suit  our  needs,  “Reversed  & Intentional” 
(Searborough,  2006). 


(Dick  & Carey,  1996,  p.  14) 


15 


"igure  A.5.4.a  “Backward”  Instructional  Design  Model 


^‘Backward^’  Instructional 
Design  Model 


Wiggins  & McTighe  (1995) 


The  program  could  be  considered  to  have  two  primary  foci,  each  comprised  of  significant 
faculty  development  components.  The  first  primary  focus  was  to  engage  professors  in  deep, 
intentional,  and  critical  reflection  through  analysis  of  the  quality  of  their  course,  teaching 
practices,  and  assessment.  Each  professor  selected  a course  to  use  as  his/her  experiment  and 
focus  for  analysis  and  redevelopment.  The  professors  were  led  through  intense  analysis  of 
that  course  and  their  role  as  professor  to  achieve  the  above  program  outcomes.  The  second 
primary  focus  engaged  the  professors  in  using  what  was  learned  from  the  critical  reflection  to 
re-design  and  re-develop  their  courses,  student  assessments,  syllabus,  and  to  broaden  their 
repertoire  of  teaching  tools  and  processes  by  choosing  additional  teaching  models  and  styles 
they  would  try  in  the  experimental  course.  They  also  structured  learning  to  broaden  student 
learning  style  options  throughout  the  learning  process.  They  redesigned  their  syllabus,  added 
a course  calendar,  and  much  more,  as  can  be  seen  in  the  descriptions  below. 


As  the  analysis  and  development  process  is  presented  below,  readers  could  become  confused 
by  the  terminology  used  to  identify  the  course  going  through  revision.  Actually,  each 
professor  was  engaged  in  the  analysis  and  revision  of  one  course  each.  However,  we  will  call 
this  same  course  two  different  terms,  the  2005  and  the  2006  course.  Throughout  the  course 
analysis  program  components  described  immediately  below,  the  focus  was  on  analyzing  the 
“existing”  or  “current”  2005  version  of  each  of  their  courses.  Once  each  professor  analyzed 
that  course  through  a series  of  analyses  and  progressed  to  the  course  development  or 
“redevelopment”  stage  in  the  program,  the  2005  course  was  redeveloped  and  improved;  it 
then  became  the  2006  course  to  be  taught  during  the  2006  experimental  research  semester. 


Analysis  of  the  2005  Course 


16 


Therefore,  we  are  discussing  one  course,  but  it  was  re-developed  and  greatly  changed 
throughout  the  faculty  development  program  process. 

Course  Knowledge  Content  Analysis 

Each  professor  analyzed  his/her  selected  2005  course  for  the  quality  of  the  knowledge 
content,  using  (Bloom’s  Knowledge  dimensions:  factual,  conceptual,  procedural,  and  meta- 
cognitive)  to  more  deeply  consider  the  knowledge  they  wanted  students  to  learn  in  the 
course.  Often  professors  and  teachers  had  difficulty  separating  knowledge  from  skills, 
education  from  training,  facts  from  information,  and  knowledge  from  other  general 
information  that  serves  as  curricular  context.  Knowledge  was  usually  considered  to  be 
fundamental  concepts,  principles,  facts,  constructs,  ideals,  etc.  that  underlie  what  might  be 
considered  broader  information.  We  decided  to  approach  it  simply,  using  the  Bloom’s 
Knowledge  dimension.  Furthermore,  professors  did  not  often  consider  knowledge  content 
priorities  or  the  amount  of  course  time  that  should  be  allocated  to  each  knowledge  piece  or 
cluster.  Priority  and  course  time  allocations  for  content  affect  more  than  teaching,  learning 
activities,  and  course  design;  the  priority  of  course  content  and  the  time  allocated  to  spend  on 
teaching  and  learning  also  affects  tests  or  other  assessments.  When  leading  professors  to 
prioritize  particular  knowledge  content  for  greater  attention  and  instructional  time,  then 
proportionally  the  same  had  to  apply  to  attention  and  time  of  the  content  on  assessments. 

Also  content  priority  could  affect  what  type  of  assessment  should  be  used  to  measure 
learning  of  particular  content.  The  professors  assigned  priority  status  to  their  knowledge 
content  at  three  levels:  primary,  secondary,  and  other  or  minor,  and  allocated  instructional 
time  appropriately.  They  also  used  these  determinations  when  building  the  tests  and 
performances  as  well  as  determining  the  other  student  assessments  or  assignments. 

The  professors  were  first  asked  to  outline  their  existing  2005  courses  for  “knowledge” 
content;  separating  “knowledge”  from  “other”  information  was  somewhat  a challenge  for 
them.  They  were  more  able  to  consider  it  from  the  perspective  of  “course  content.”  Also  it 
cannot  be  assumed  that  engineering  and  technology  professors  generally  possess  traditional 
outlining  skills,  especially  if  English  is  a second  language  and  culture  or  language  logic  is 
different.  When  asking  a diverse  group  to  outline  their  courses  by  primary,  secondary,  and 
other  minor  but  important  knowledge  or  content,  we  found  it  was  not  an  easily  accomplished 
task.  We  did  finally  arrive  at  a point  where  each  professor  had  a “list”  of  important 
knowledge  content,  which  they  itemized  by  priority:  (1)  “primary”  or  must  include  and  spend 
the  greatest  amount  of  time  on;  (2)  “secondary”  or  important  and  must  spend  significant  time 
on;  and  (3)  “other”  important  knowledge  that  needs  to  be  acknowledged  or  mentioned,  but 
not  much  time  spent  on  it,  and/or  assign  students  to  learn  it  on  their  own  or  outside  of  class 
time  but  still  hold  them  accountable  for  it.  Simply,  the  professors  outlined  or  listed  course 


2 

This  was  the  first  event  where  the  “time”  aspect  of  a course  or  a course  calendar  was  mentioned.  None  of  the 
professors  included  a course  calendar  in  their  syllabus  so  students  could  see  what  course  content  and  activities 
would  be  addressed  each  day  or  week  or  how  long  they  would  spend  on  it.  Nor  were  many  due  dates  firmly 
established  or  mentioned  at  all  on  most  of  the  syllabi.  Professors  did  not,  themselves,  seem  to  have  any  real 
sense  of  “knowledge  content  and  time”  for  their  courses.  Of  course,  they  had  some  intuitive  sense  (somewhat 
operational)  about  what  they  wanted  to  cover  overall,  but  there  was  no  “real”  sense  of  the  timeline  tied  to 
knowledge  content,  course  activities,  due  dates  for  assignments,  etc.  There  is  more  about  this  later  when  the 
professors  reached  the  point  of  developing  a course  calendar  to  include  in  the  2006  syllabi. 
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content,  broke  it  out  into  at  least  two  levels  (but  most  by  three  levels),  and  then  prioritized 
and  considered  it  regarding  the  time  factor.  Finally,  most  of  them  identified  the  course 
content  by  Bloom’s  Knowledge  dimensions:  factual,  conceptual,  proeedural,  and/or  meta- 
eognitive  for  the  purpose  of  making  instruetional  deeisions  about  more  effeetive  teaehing 
strategies  (Wiggins  & MeTighe,  2005).  The  simple  worksheets  below  were  used  to  organize 
the  initial  knowledge  content  analyses  (See  Figures  A. 5. 5 and  A. 5. 6).  Also  when  knowledge 
content  is  balaneed  aeross  all  of  Bloom’s  Knowledge  dimensions,  then  there  is  the  great 
possibility  of  aehieving  student  learning  at  the  higher  levels  of  Bloom’s  eognitive  proeess 
dimensions. 


Figure  A. 5. 5:  Discipline  Course  Outline  (See  Seetion  Cl  for  worksheet) 


Course  Disciplinary 

Science(s) 

Mathematics 

Communication 

Knowledge  Content 

Foundation  Required 

Foundation  Required 

Skills  Required 

(Id.  Major  and  minor 

(Id.  Physics,  Chemistry, 

(Id.  Algebra,  Trig, 

(Id.  written,  spoken. 

Concepts,  principles, 

Biology,  etc.) 

Calculus,  Differential 

graphical,  listening) 

Facts,  information,etc.) 

Equations,  etc.) 

technical,  etc.) 

I.  Unit 

A. 

A. 

A. 

A. 

1. 

1. 

1. 

1. 

2. 

2. 

2. 

2. 

B. 

1. 

2. 

II.  Unit 

Figure  A. 5. 6:  Course  Knowledge  Content  Priorities 


Knowledge  Primary-major  Secondary  Needs  to  be 

addressed 

Content  (Allocate  most  time)  (Allocate  significant  time)  (Mention  briefiy- 

assign) 

A. 

1. 

2. 

B. 

1. 

2.  etc. 
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Figure  A. 5. 7:  Clarifying  Curricular  Content  Priorities  and  Assessment  Methods 

Assessment  Methods  Knowledge  and  Skills 


Course  Mathematics,  Science,  and  Communication  (MSC)  Analysis 
Another  aspect  of  analyzing  the  course  knowledge  content  occurred  from  the  mathematics. 
science,  and  communication  perspectives.  Each  professor  considered  his/her  knowledge 
content  priority  lists  from  these  three  perspectives  and  listed  or  outlined  the  necessary  or 
expected  pre-requisites  or  co-requisite  knowledge  or  skills  embedded  within  the  engineering 
and  technology  knowledge  content.  This  led  us  into  consideration  of  what  general  education 
knowledge  and  competencies  would  be  expected  of  students  if  they  were  to  be  successful  in 
these  courses  (we  used  the  NIU  general  education  books).  Illuminating  those  expectations 
was  important  so  the  professors  could  really  focus  on  what  they  expected  of  students  and, 
ultimately,  what  they  expected  of  themselves  and  their  own  address  of  knowledge  content  in 
the  course,  discipline-  or  general  education-based.  This  was  beneficial  and  helped  them  to 
really  flush  out,  or  confirm,  the  top  priority  knowledge  content  for  the  course  from  both 
discipline  content  and  the  general  education  content,  especially  math,  science,  and 
communication  preparation  perspectives.  Although  usually  consideration  of  the 
ABET(EAC/TAC)/NAIT  standards  or  learning  outcomes  would  come  first,  because  our 
programs  were  already  accredited,  the  courses  (or  course  objectives)  were  already  linked  to 
the  ABET/NAIT  standards  or  learning  outcomes.  Therefore,  the  strength  of  the  courses  in 
achieving  those  outcomes  and  the  strength  of  the  connections  were  the  foci  of  our  analysis.  It 
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was  important  that  the  professors  grasped  what  was  expected  from  the  course  in  its  current 
state  (2005).  A clearer  focus  on  the  knowledge  content  and  its  breakout  of  major,  secondary, 
and  minor  content  and  the  priority  of  those  content  clusters  were  our  first  tasks  in  the  analysis 
process.  The  professors  completed  the  outlines  and  identified  the  inherent  mathematics, 
science,  and  communication  foundation  knowledge  and  skills  expected  of  students  upon 
entering  the  course.  This  was  one  milestone  in  the  analysis  process. 

(NIU  General  Education  Goals-  http://www3.niu.edU/provost2/facpers/appm/IIID.l.htm). 

Course  Knowledge  Source 

Finally,  the  professors  identified  the  knowledge  content  sources  as  the  text,  themselves, 
manuals,  tutorials,  field,  speakers,  literature,  etc.  This  analysis  led  them  to  understand  they 
relied  much  too  heavily  on  the  textbook,  and  in  some  cases  where  they  were  the  source,  they 
determined  they  were  not  really  using  their  expertise  as  well  as  it  could  be  used.  Almost  no 
external  sources  were  used  for  knowledge  content  (e.g.,  field,  literature,  case  studies) 

(Bloom,  1956). 

Teaching  Models,  Styles  and  Student  Learning  Styles  (Portfolio  Component  Section  B5) 
Following  the  outlining  of  the  course  content  knowledge,  the  determination  of  content 
priority,  the  math,  science,  communication,  and  source  analyses,  each  professor  focused  on 
what  teaching  models  and  teaching  styles  were  primarily  used  and  then  what  student  learning 
styles  they  felt  students  had  the  opportunity  to  engage  in  throughout  the  course.  Each 
professor  considered  his/her  course  content  and  identified  the  teaching  models,  styles,  and 
student  learning  styles  he/she  believed  best  described  what  occurred  during  the  teaching  and 
learning  process  throughout  their  courses.  To  guide  them  in  this  process,  a list  of  teaching 
models,  teaching  styles,  and  learning  styles,  followed  by  brief  definitions,  descriptions,  or 
explanations  were  provided.  In  some  cases,  two  versions  were  provided;  for  example,  both 
Kolb  and  Felder  learning  styles  were  presented.  They  quickly  discovered  their  courses  were 
primarily  lecture  based,  although  there  were  learning  events  throughout  most  of  the  courses 
during  which  the  inquiry,  training,  simulation,  and  other  models  were  used,  just  not  often 
enough.  Clearly,  they  discovered  a need  to  diversify  the  teaching  models  used  throughout  the 
courses  as  well  as  a need  to  diversify  styles.  The  same  trend  was  obvious  when  considering 
what  learning  styles  they  felt  their  teaching  made  possible  for  students.  Although  there  were 
particular  learning  events  where  students  could  use  a variety  of  learning  styles,  the  professors 
discovered  they  generally  taught  in  a way  that  limited  student  learning  style  diversification 
and  the  ability  for  students  to  expand  their  comfort  zone  across  multiple  learning  styles. 

Professors  had  not  designed  their  instruction  or  courses  to  intentionally  use  a variety  of 
teaching  models  and  styles  or  to  engage  students  across  the  range  of  learning  styles. 
Therefore,  they  were  possibly  preventing  some  students  from  learning  as  well  as  might  have 
been  possible  with  a broader  range  of  learning  style  opportunities  (Anderson  & Krathwohl, 
2001;  Bloom,  1956;  Dale,  1969;  Felder,  1988;  Grasha,  1996;  Joyce,  Weil,  & Calhoun,  2004; 
Kolb,  1984;  Mosston  & Ashworth,  1990).  Generally,  there  was  a clear  need  to  diversify 
teaching  models  and  styles  so  students  could  use  their  primary  learning  style  while  also 
expanding  to  become  comfortable  with  other  learning  styles. 
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Knowledge  Content  Considered  by  Bloom’s  Taxonomy  and  Dale’s  Cone  of  Learning 
(Portfolio  Course  Analysis  Section  B5) 

The  analysis  process  continued  with  Dale’s  Cone  of  Learning  to  determine  the  range  of 
passive  versus  active  learning  and  Bloom’s  traditional  and  revised  Taxonomies  of  Learning 
(professor’s  choice)  - the  Cognitive  Process  Dimension  - to  determine  how  often  students 
were  provided  the  opportunity  to  engage  in  higher  levels  of  cognitive  processes  and  critical 
thinking  while  learning  (or  were  they  just  memorizing?).  This  led  the  professors  to  consider 
the  level  of  critical  thinking  stimulated  by  their  teaching  or  required  in  course  activities  and, 
finally,  to  determine  whether  they  thought  their  course  was  teacher,  knowledge,  assessment, 
or  learner  centered.  We  provided  simple  handouts  with  definitions,  descriptions,  or  examples 
to  become  familiar  with  Bloom  and  Dale’s  models,  using  the  models  as  metrics  to  analyze 
the  courses.  Questions  or  more  explanation  was  provided  when  requested  or  needed. 
Although  this  might  be  perceived  by  readers  as  a superficial  analysis  (and  in  some  ways  it 
was),  it  worked  very  well  to  stimulate  the  professors  to  an  awareness  level  of  where  student 
learning  was  on  Bloom’s  Cognitive  Process  Dimension,  whether  students  were  actively  or 
passively  engaged,  whether  they  were  requiring  critical  thinking,  and  to  what  level,  their 
courses  were  teacher,  knowledge,  assessment,  or  learner  centered.  As  they  redeveloped  their 
courses,  a deeper  understanding  occurred  as  a result  of  their  use  of  Bloom  and  Dale  models 
as  a basis  for  course  design  (Anderson,  Cruikshank,  Mayer,  Pintrich,  Raths,  & Wittrock, 
2001;  Bloom,  1956;  Dale,  1964;  Nosich,  2005). 

Dale’s  Cone  of  Learning  was  used  as  a simple  and  informal  guide  to  which  they 
could  compare  and  consider  the  level  of  active  versus  passive  learning  occurring  in 
the  courses  of  focus.  Then  they  could  determine  where  they  needed  to  make  the 
student  learning  more  active.  They  found  their  courses  to  be  mostly  passive;  some 
had  more  active  segments,  but  generally,  most  of  the  learning  was  taking  place 
passively  by  students. 

Bloom’s  Taxonomy  of  Learning  (traditional  or  revised)  - the  Cognitive  Process 
Dimension  - was  used  also  as  a guide  from  which  to  judge  the  critical  thinking  and 
levels  of  cognitive  processing  students  were  achieving  in  the  courses.  During  this 
analysis,  the  professors  varied  in  their  self-reported  judgments  of  the  level  of  learning 
occurring  by  their  students.  Some  reported  that  the  course  and  student  learning  were 
mostly  at  the  lower  end  of  Bloom’s  Taxonomy,  regardless  of  which  version  was  used. 
Others  reported  that  students  were  learning  at  the  upper  levels  of  Bloom’s;  however, 
in  examining  and  analyzing  the  course  content,  assessments,  learning  activities,  they 
relied  mostly  on  mathematically-oriented  problem  solving  with  no  rubrics  or  criteria 
available  to  guide  or  score  performance.  The  problems  were  poorly  structured, 
grading  was  very  suspicious  and  far  too  subjective,  and  there  were  too  few 
opportunities  for  students  to  provide  evidence  of  learning  and  too  few  methods  to 
provide  students  with  opportunities  to  show  evidence  of  learning.  So  the  professors’ 
self-reported  responses  and  lack  of  awareness  of  what  higher  levels  of  cognitive 
process  really  entailed  resulted  in  somewhat  biased  and  uninformed  analyses. 
However,  this  was  an  excellent  method  for  them  to  learn  about  the  higher  levels  of 
cognitive  processes  they  could  achieve  with  their  students.  There  were  two  professors 
who  possibly  had  some  significant  higher  levels  of  learning  occurring,  but  not 
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formally  structured,  based  upon  a review  of  the  course  materials.  And,  although  this 
was  true,  when  formalized,  their  courses  and  potential  student  learning  were 
strengthened  greatly. 

Critical  thinking  was  subjectively  addressed  by  continuing  from  the  Bloom’s  analysis 
to  judging  whether  the  professors  thought  critical  thinking  was  evident  at  the  low, 
medium,  or  high  ranges  in  their  courses  or  student  learning.  Two  professors  did  not 
complete  this  component,  and  their  Bloom’s  analysis  would  have  revealed  that  not 
much  critical  thinking  was  occurring.  The  others  reported  a range  of  1 - 4,  with  5 
being  very  high  (or  low  to  high).  However,  once  again,  in  reviewing  their  courses, 
assessments,  and  learning  activities,  this  self-report  was  biased  towards  the  higher 
ends.  The  analyses  provided  evidence  that,  with  some  exceptions,  there  were  mostly 
lower  levels  of  critical  thinking  being  required  of  their  students. 

The  professors  considered  what  their  courses  were  centered  on  and  judged  them  to  be 
teacher  centered,  knowledge  centered,  assessment  or  student  centered.  In  other  words, 
the  burden  of  learning  was  on  the  teacher,  not  on  the  students  where  it  should  be,  and 
the  assessments  were  poor.  They  realized  that  they  needed  to  create  their  re- 
developed courses  so  the  “centeredness”  shifted  appropriately,  where  there  would  be 
overlap  or  integration  of  the  four  types  of  “centeredness”  (teacher,  knowledge, 
assessment,  learner)  but  also  to  design  their  courses  to  be  primarily  learner  centered. 

The  course  analysis  revealed  that  professors  needed  to  formally  design  learning  so  it  was 
more  active  and  required  students  to  engage  at  the  higher  levels  of  Bloom’s  Cognitive 
Process  Dimensions  and  critical  thinking.  The  process  also  revealed  that  professors  needed  to 
develop  clear  criteria  for  student  performance  and  for  judging  evidence  of  learning  and  that 
what  was  to  occur  in  the  course  and  student  learning  was  intentional  (what  the  professors  had 
clearly  determined  as  important  to  learn)  and  could  be  supported  by  formal  evidence  of 
student  learning.  (Anderson  & Krathwohl,  2001;  Bloom,  1956;  Dale,  1969;  Frye  et  ah,  2003; 
Kolb,  1984;  Nosich,  2005).  The  following  worksheet  was  used  as  a tool  to  organize  the  initial 
analysis. 
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Table  A.5.^^ 

1:  Content  Schedule  and  Teaching  Models,  Sty] 

ies,  and  Bloom's  Analysis 

w 

e 

e 

k 

Content 

Topic: 

Bloom’s 

Knowledge 

Dimension 

Factual, 

Conceptual, 

Procedural, 

Meta- 

cognitive 

Content 

Source 

Text, 

Professor, 

Field, 

Literature, 

Speaker, 

DVD, 

etc. 

Teaching 

Style 

a-k 

Teaching 

Model 

1-24 

name 

Kolb 

Learning 

Style 

CE,  AE, 
AC,  RO 

Dale's 

Cone 

Active 

or 

Passive 

Bloom's 

Cognitive 

Process 

Dimension 

Traditional: 

Evaluation, 

Synthesis, 

Analysis, 

Application, 

Comprehension 

Knowledge 

Bloom's 

Cognitive 

Revised: 

Create, 

Evaluate, 

Analyze, 

Apply, 

Under 

stand. 

Remember 

Critical 

Thinking 

Low 

Medium 

High 

Centered? 

Teacher, 

Knowledge 

Assessment, 

Learner 

1 

2 

Note:  a two  or  three  day/week  form  was  used.  (See  Section  Cl  for  worksheet) 


Gaps  Analysis  Summary  (See  Portfolio  component) 

This  analysis  stemmed  from  the  content  analysis  and  teaching/leaming  analyses  above;  the 
form  was  an  attempt  to  summarize  what  they  had  observed  about  their  2005  courses  and 
teaching  practices  so  they  could  better  view  the  whole  picture.  In  addition,  it  led  them  to  an 
analysis  of  whether  they  felt  the  ABET(EAC/TAC)/NAIT  outcomes  were  being  achieved 
from  the  perspective  of  their  course  outlines  or  knowledge  content  and  priority  lists.  They 
simply,  once  again,  reviewed  their  course  outlines  and  general  education  expectations  from 
the  above  analyses  to  determine  how  well  the  ABET(EAC/TAC)/NAIT  outcomes  were  being 
covered.  Then  they  reviewed  their  2005  student  learning  objectives  from  the  perspective  of 
teaching  models,  styles,  learning  styles.  Bloom’s  Knowledge  and  Cognitive  Dimensions,  and 
Dale’s  Cone  of  Beaming.  The  professors  used  the  notations  of  or  “C”  on  the  form  to 
identify  whether  they  felt  they  were  “okay”  or  needed  to  “consider”  changing  something  or 
trying  a new  method.  This  worksheet  was  somewhat  duplicative  but  added  analysis  of  the 
course  objectives  or  student  learning  outcomes  and  provided  a more  complete  picture  - one 
that  was  more  useful  as  it  clearly  revealed  what  professors  needed  to  consider  addressing 
when  redeveloping  their  2005  courses  and  making  instructional  decisions  for  the  redeveloped 
2006  courses.  Clearly,  the  GAPS  between  where  they  were,  the  “current  reality”  of  the  2005 
courses,  and  what  they  could  move  towards,  “their  vision”  for  the  2006  courses,  were 
revealed.  The  following  worksheets  were  used  to  organize  and  summarize  the  teaching  and 
learning  factors  described  above. 

Once  the  GAPS  Analysis  Summary  was  completed  and  the  professors  continued  on  to  the 
course  redevelopment  program  component  at  a later  point  in  the  program,  another  more 
integrated  form  was  used  where  the  ABET(EAC/TAC)/NAIT  Outcomes  are  listed  in  the  left 
column.  The  General  Education  Goals  are  shown  embedded  in  each  outcome,  a more 
integrated  format.  That  format  is  shown  later. 
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Table  A. 5. 5:  GAPS  Analysis  Summary  (See  Portfolio  Seetion  B.5  - Report  and  Faeulty  Examples) 

NOTE:  Each  faculty  member  chose  which  Standards  or  Outcomes  form  below  to  use  depending  upon  whether  they  were  accredited  by  ABET  Engineering,  ABET/ 
EAC/TAC  for  Engineering  Technology,  or  NAIT  for  Technology.  Our  college  has  all  three  accreditations.  ABET/EAC/TAC  and  NAIT  have  been  merged  for  use  by 
the  Department  of  Technology  with  both  Engineering  Technology  and  Technology  programs.  Legal  size  paper  works  best  for  workshop  use.  (See  Section  Cl  for 
worksheet) 


GAPS  Analysis  Summary; ABET -Engineering  Outcomes 


a. 

apply 

math, 

science, 

engineer- 

ing 

b. 

design/ 

conduct 

experi- 

ments; 

analyze, 

interpret 

data 

c. 

design 

system, 

compo- 

nent, 

process- 

given 

constrain 
ts,  etc. 

d. 

function  on 
interdis- 
ciplinary 
teams 

e. 

identify, 
formulate, 
solve  engi- 
neering 
problems 

f. 

understand 

professional, 

ethical 

responsibility 

g- 

ability  to 

communicate 

effectively 

h. 

understand 
impact  of 
engineering 
Solve  global 
economic, 
environmental, 
issues  for 
society 

i.  recognition 
of  need  for, 
and  ability 
to  engage  in 
life-long 
learning 

j- 

Knowledge 

in 

contem- 

porary 

issues 

k.  ability 

to  use 

techniques, 

skills,  and 

modern 

engineering 

tools 

Professors 

identify 

St.  L. 
Outcome 

for  each 

standard. 

Those  that 

are  not 

addressed 

by  the  course 

are  left  blank. 

GAl 

PS  Analysis  Summar 

ABET(] 

EAC/TA 

C)/NAIT-Engineering  Technology  and  Industrial  Technology  Outcomes 

a. 

mastery 
of  know- 
ledge, 
tech- 
niques, 
skills, 
modern 
tools 

b. 

ability 
to  apply 
current 
know- 
ledge; 
adapt  to 
emerging 
applica- 
tions of 
math, 
science, 
techno- 
logy 

c. 

ability  to 
conduct, 
analyze, 
interpret 
experi- 
ments; 
apply 
experi- 
mental 
results 
to  improve 
processes 

d. 

ability  to 
apply 
creat- 
ivity in 
design  of 
systems, 
compo- 
nents, 
processes 

E. 

ability 

to 

function 

effect- 

tively 

on 

teams 

f. 

ability 

to 

identify, 

analyze, 

solve 

technical 

problems 

g- 

ability 

to 

com- 

municate 

effect- 

ively 

writing 

h. 

ability 

to 

commu- 

nicate 

effect- 

ively 

orally 

recognize 
need  for, 
ability  to 
engage  in 
lifelong 
learning 

j- 

ability 

to 

under- 

stand 

profess- 

sional, 

ethical, 

social 

respon- 

sibilities 

k. 

respect  for 
diversity; 
knowledge 
of  contem- 
porary 
profess- 
sional, 
societal, 
global 
issues 

1. 

commit 
to  quality, 
timeliness, 
continuous 
improve- 
ment 

m. 

ability  to 

program 

computers 

and/or  use 

computer 

applications 

effectively 

n.  ->q. 
ability  to 
use 

modern 

laboratory 

techniques, 

skills, 

equipment 

effectively 

Refer  to 
ABET(EAC 
/TAO/NAI 
Tfor 

o, p,q 

Profs 

identify 

SL  Outcomes 

for  each 

standard. 

Those 

that  are 

not 

addressed 

by  the 

course 

are 

left 

blank. 

GAPS  Analysis  Summary;  NIU  General  Education  Goals 


St.  L. 
Outcome 

Writing 

Speaking 

Listening 

Quantitative 

Reasoning 

Use  of 

Resources- 

Technology 

His/herstorical 
Development 
Of  Culture 

Significance 
of  Arts 

Cultural 

Traditions; 

Philosophical 

Ideas 

Methods  in 
Science 
Methods  in 
Social  Science 

Interrelatedness 

Across 

Disciplines 

Social 

Responsi- 

bility 

Professors 

respond 

on 

following 

rows. 

Each  learning 

outcome 

Vertically 

is  addressed 

for  each  column. 
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Table  A. 5. 5:  GAPS  Analysis  Summary  (continued) 

Note:  The  student  learning  outcomes  by  teaching  model  worksheet  below  is  laid  out  in  one  row  on  a legal  sized  form  so  that  student  learning  outcomes  can  be  listed 
in  far  left  column  and  then  which  teaching  models  used  checked  off  horizontally  for  each  outcome.  Parts  1 and  2 for  Models  fit  on  legal  sized  paper. 


Student  Learning 

Objectives/Outcomes  & Teaching  Models 

Part  1) 

St.  L. 
Outcome 

Memory 

Progressive 

Part 

Advanced 

Organizer 

Lecture 

Reciprocal 

Teaching 

Mastery 

Learning 

Cooperative 

Learning 

Graphic 

Organizers 

Concept 

Attainment 

Synectic 

Psycho- 

motor 

Meta- 

phore 

Professors 

respond 

on 

following 

rows. 

Each 

learning 

outcome 

vertically 

is  addressed 

for 

each 

column. 

Student  Learning  Objectives/Outcomes  & Teaching  Models  (Part  2) 


St.  L. 
Outcome 

Concept 

Formation 

Concept 

Presentation 

Conceptual 

Induction 

Deduction 

Inquiry 

Simulate 

Jurisprudential 

Direct 

Instruction 

Training 

Non- 

direct 

Role 

Professors 

respond 

on 

following 

rows. 

Each 

learning 

outcome 

Vertically 

is  addressed 

for 

each 

Column 

S 


udent  Learning  Objectives/Outcomes  & Teaching  Styles 


St.  L.  Outcomes 

Command 

Practice 

Reciprocal 

Self- 

Check 

Inclusion 

Guided 

Discovery 

Convergent 

Discovery 

Divergent 

Production 

Learner 

Designed 

Learner 

Initiated 

Self 

Teaching 

Professors 

respond 

On 

following 

rows. 

Each 

learning 

outcome 

vertically  is 

addressed 

for  each 

column. 

Student  Learning  Objectives/Outcomes  & Kolb’s  Learning  Styles 


St.  L.  Outcomes 

Concrete  Experience 

Abstract  Conceptualization 

Active  Experimentation 

Reflective 

Observation 

Professors  respond  on  following  rows. 

Each  learning  outcome 

vertically  is  addressed 

for  each  column. 

Student  Learning  Object 

tives/Outcomes  & Bloom’s  Taxonomy  - Cognitive  Dimensions  & Da 

le’s  Cone 

St.  L.  Outcomes 

Dale's  Cone 
Levels  P A 
A+ 

Knowledge 

Remember 

Comprehension 

Understand 

Application 

Apply 

Analysis 

Analyze 

Synthesis 

Evaluate 

Create 

Critical 
Thinking 
Level  L M H 

Professors  respond  on  following 

Rows. 

Each  learning 

Outcome 

vertically 

is  addressed 

for  each 

column. 

(Scarborough,  2006) 
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Instructional  Design  GAPS  Analysis 

Although  similar  or  slightly  redundant  but  adding  pieces  to  analyze,  the  following  form  and 
process  moved  the  professors  forward  to  include  and  analyze  their  student  assessments 
beside  the  student  learning  outcomes  and  against  Bloom’s  Cognitive  Dimension  and  Dale’s 
Cone  of  Learning.  This  aspect  of  the  process  was  used  to  engage  them  in  identifying  or 
matching  the  student  assessments,  mostly  tests  for  the  2005  course,  to  the  student  learning 
outcomes.  This  process  engaged  them  in  connecting  the  knowledge  content  they  felt  they 
were  teaching  to  their  tests  or  other  assessments  and  revealed  more  clearly  what  the 
professors  were  actually  testing  or  measuring.  Linking  tests  and  test  items  directly  to 
outcomes  informed  the  professors  whether  they  were  testing  what  they  thought  was  being 
tested.  The  relationship  between  what  is  taught  and  what  is  tested  has  long  been  a problem  in 
courses  or  classrooms,  even  for  experienced  education  professionals.  Are  we  really  testing 
what  we  think  we  are?  A complaint  often  heard  from  students  is  that  the  professors  did  not 
cover  what  was  on  the  test,  that  they  did  not  test  what  they  said  they  were  going  to,  or  that 
the  tests  did  not  correspond  with  course  content  being  taught  - from  the  students’ 
perspective.  This  particular  process  also  asked  the  professors  to  repeat  by  outcomes.  Bloom 
and  Dale  abbreviated,  and  the  knowledge  content  sources  (e.g.,  text,  professor,  etc.).  The 
reason  for  the  abbreviated  repetition  was  so  the  linking  would  flow  logically  and  serve  as  a 
check  and  balance.  It  is  helpful  when  professors  also  identify  the  source  of  the  content  for  the 
tests  and  test  items  or  other  assessments.  That  is  how  the  GAPS  are  identified  and  how  the 
missing  links  between  what  is  being  taught  and  tested  are  revealed.  If  content  cannot  be 
mapped  directly  to  assessments,  to  tests  and  test  items,  or  to  performances  and  criteria  on  the 
rubrics,  then  a gap  or  missing  link  reveals  itself.  This  is  truly  illuminating  for  professors. 
Mapping  is  a great  process  for  this  and  many  other  analyses. 

As  we  progressed  through  the  analysis  process,  the  overlap  (or  intended  redundancies) 
gradually  resulted  in  greater  familiarity  and  understanding  of  the  models,  styles,  outcomes, 
and  instructional  issues.  As  we  progressed,  we  were  adding  factors  for  consideration.  The 
professors  gradually  realize  that  teaching  and  learning  are  complex  constructs,  yet  small 
changes  can  make  great  differences.  This  expands  and  deepens  understanding  gradually. 

Finally,  this  program  component  asked  them  to  consider  where  they  thought  their  tests  and 
test  items  were  on  Bloom’s  Cognitive  Dimension  and  Dale’s  Cone  and  revealed  where  they 
needed  to  make  changes  so  students  could  better  achieve  learning  at  the  higher  cognitive 
levels  and  in  more  active  ways  (even  on  traditional  tests).  It  is  important  to  ensure  that  tests 
or  assessments  are  also  achieving  higher  cognitive  dimensions  and  are  as  active  as  possible. 
Although  a few  professors’  analyses  revealed  learning  to  be  intermittently  at  higher  levels  of 
learning  on  Bloom’s  and  more  active  on  Dale’s,  there  were  many  opportunities  to  improve. 
There  were  no  formal  structures  and  grading  criteria  (rubrics),  and  the  relationships  or  links 
between  objectives  or  student  learning  outcomes,  tests,  and  test  items  were  unclear  and  often 
weak.  This  form  and  process  also  asked  them  to  analyze  any  non-test  assessments,  (e.g., 
projects,  performances).  A few  professors  required  projects  of  the  students,  but  the  grading 
was  very  subjective  with  no  established  criteria  or  rubrics.  Also  there  were  no  formal 
performance  tasks  identifying  what  students  were  expected  to  do.  As  is  typical  in  many 
courses,  a project  topic  was  provided  and  then  students  or  student  groups  took  it  from  there. 
Also  missing  were  any  criteria,  expectations,  or  consideration  of  how  to  grade  team  work. 
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Team  work  was  not  formally  set  up  so  individual  and  group  learning  accountability  was 
accomplished.  Thus,  the  objective/outcome-to-test-and-item  matching  (the  assessment,  tests 
and  projects  measured  against  Bloom’s  Cognitive  Dimension  and  Dale’s  Cone)  revealed 
many  opportunities  to  strengthen  the  quality  and  connections  of  assessments.  Clearly,  the 
professors  needed  to  move  toward  an  assessment  plan  that  was  more  multifaceted  if  they 
were  to  create  a broader  range  of  opportunities  for  students  to  provide  evidence  of  learning 
that  corresponded  with  a broader  range  of  learning  styles  and  performance  styles  (Bloom, 
1956;  Johnson  & Johnson,  1998).  The  following  worksheet  was  used  to  organize  this 
analysis. 


Table  A. 5. 6:  Instructional  Design  GAPs  Analysis  - Connecting  learning  outcomes  to  assessments 


(See  Portfolio  C 


for  worksheets) 


ABET 

Student 

Bloom/Dale 

Knowledge 

Student 

Bloom/Dale 

Tests 

Bloom/Dale 

Performance 

Bloom/ 

Outcome 

Learning 

Sources 

Assessments 

Dale 

a-k 

Outcome 

Evaluation/ 

Evaluation/Active 

and 

Evaluation/A 

If  Any; 

Evaluation/ 

ENG 

(s) 

Active 

Professor, 

listed  on 

Synthesis/Active 

Synthesis/A 

Active 

Synthesis/ 

Text, 

syllabus 

Analysis/Active 

Items 

Analysis/A 

If  none, 

Synthesis/ 

ABET/ 

listed  on 

Active 

Cases, 

Application/Active 

Application/A 

leave 

Active 

TAG/ 

syllabus 

Analysis/ 

Field, 

Comprehension/P 

Comprehension 

blank 

Analysis/ 

NAIT 

Active 

Literature 

Knowledge/Passive 

/P 

Active 

Standard 

Application/ 

Speaker, 

Knowledge/P 

Application/ 

A-Q 

Active 

References, 

Active 

Tech 

Comprehen- 

etc. 

Compre- 

sion/Passive 

hension 

Knowledge/ 

/Passive 

Passive 

Knowledge/ 

Passive 

Test  Analysis 

Once  the  overall  analysis  of  the  quality  of  the  tests  and/or  other  student  assessments 
regarding  the  connections  to  the  student  learning  outcomes  was  completed,  we  then  formally 
analyzed  their  “current”  2005  tests.  In  reviewing  the  existing  tests/exams  for  their  courses,  as 
expected  there  was  a wide  variety  of  test  types.  Most  tests  involved  problem  solving  rather 
than  totally  objective  items;  some  had  multiple  choice  items  involving  problems  that  had  to 
be  solved  before  choosing  a response;  and  some  had  word  problems  where  formulas  had  to 
be  chosen  and  then  solved.  Some  tests  were  in-class,  while  others  were  take  home  tests. 

Some  tests  included  open-ended  problems,  not  prompted  with  responses,  and  some  were 
open-ended,  short  answer  items.  The  assessments,  mostly  tests,  were  not  “problem  based.” 
The  literature  establishes  a difference  in  problem  based  learning  versus  problem  solving. 
Problem  based  learning,  which  can  also  be  an  assessment,  is  more  open  ended;  the  answers 
or  responses  are  not  predetermined  as  didactic.  Typically,  problem  solving  engages  the 
student  in  solving  a problem  for  which  the  professor  is  expecting  a particular  or 
predetermined  answer.  Regarding  quality,  the  range  was  varied  across  professors  and  within 
and  across  the  tests.  It  is  clear  that  the  professors  were  trying  to  engage  students  in  problem 
solving  rather  than  memorization  for  the  most  part.  That  was  positive.  The  quality  of  items 
ranged  both  within  any  single  test  and  across  the  tests  as  well.  Although  various  grading 
procedures  were  used,  the  grading  was  very  subjective  for  the  open-ended  and  problem 
solving  items.  There  were  no  rubrics  or  criteria  to  use  when  judging  student  solutions. 
Therefore,  awarding  partial  points  for  an  overall  incorrect  item  was  subjective  and  not 
perceived  as  consistent  across  student  tests.  The  professors’  discussion  of  grading  procedures 
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led  us  to  believe  that  it  was  very  important  for  them  to  analyze  their  grading  proeedures  and 
structure.  They  needed  to  identify  definitive  criteria,  structure,  and  formal  reasoning  for 
giving  points  and  then  confirm  that  they  must  stick  to  their  grading  procedures  once 
established.  The  discussion  was  very  interesting  with  all  aspects  of  arguments,  some  valid, 
some  definitely  not  valid,  especially  those  related  to  “curving”  grades.  We  had  to  dispel 
myths  about  the  benefits  and  inappropriateness  of  “curving”  grades  (Anderson  & Speck, 

1998;  Dominowski,  2001;  Frye,  1994;  Royse,  2001;  About  Grades,  n.d.). 

An  important  aspect  of  this  program  component  was  the  week  devoted  to  test  analysis  and 
test  development.  The  professors  learned  how  to  analyze  tests  using  their  current  tests  where 
possible;  however,  when  that  was  not  possible,  because  tests  were  problem  based  and  not 
objective,  then  they  were  provided  a (real  test)  case  for  the  learning  process.  Each  professor 
learned  to  analyze  objective  tests.  This  performance-based  learning  process  was  the 
introduction  to  the  development  of  good  or  improved  objective  tests.  Each  professor  learned 
the  purpose  of  item  analysis  and  the  factors  affecting  item  statistics,  and  then  engaged  in 
analyzing  for  item  difficulty  and  item  discrimination.  They  discussed  the  “flagged”  items  and 
made  decisions  about  how  to  address  them.  They  considered  the  validity,  reliability,  and 
standard  error  of  measurement  regarding  the  tests  analyzed  and  ended  the  analysis  sessions 
with  a better  understanding  of  what  caused  tests  to  be  of  higher  or  lesser  quality.  Ultimately, 
they  realized  that  test  analysis  serves  as  a diagnostic  tool  for  the  ongoing  quality 
improvements  of  tests  and  instruction.  Einally,  they  were  introduced  to  NIU’s  Testing 
Services  and  what  that  service  could  offer  them.  This  session  segued  into  new  test 
development  (Nitko,  2004).  See  the  full  Report  on  Tests  in  Section  B6. 

Course  Analysis  Completion 

The  test  analysis  program  component  ended  the  2005  course  analysis  segment  of  the  faculty 
development  program.  However,  it  did  not  end  the  process  of  analysis.  Professors  continued 
to  be  analytical  throughout  the  critical  reflection  process,  thus  becoming  more  intimate  with 
the  inherent  process  of  “Reflective  Practice,”  a foundation  for  the  Scholarship  of  Teaching 
and  our  faculty  development  program. 

2006  Course  Development 
(2005  Course  Redevelopment) 

The  course  redevelopment  process  was  based  upon  the  2005  course  analysis  results  described 
above  and  reported  in  the  college  portfolio  results  section  - B5  and  B6.  Each  professor 
identified  changes  to  make  in  the  redeveloped  course  based  on  what  was  learned  through  the 
above  analyses.  In  beginning  the  formal  redevelopment  of  their  2005  courses  and  in  using  the 
Reversed  Design  and  our  Intentional  Model,  they  began  with  the  identification  of  the  student 
learning  outcomes  and  connecting  to  the  ABET  and  NAIT  outcomes.  This  was  followed  by 
the  development  of  new  tests  with  both  objective  and  problem  solving  items;  development  of 
new  and  formal  performance  tasks  and  corresponding  rubrics;  scoring  the  grading 
procedures.  Those  were  followed  by  making  instructional  decisions  about  teaching  models, 
styles,  and  student  learning  styles  and  culminated  in  the  redevelopment  of  the  course  syllabus 
and  making  other  instructional  decisions.  Each  is  described  below. 
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Student  Learning  Outcomes  Development 


Learning  Objective  or  Outcomes  Analysis  (See  Portfolio  - Section  B7  and  PowerPoint 
Presentation  - Section  D3) 

From  this  point  in  the  program,  the  focus  was  development  or  “redevelopment”  of  the 
selected  2005  courses.  The  courses  were  considered  from  the  perspectives  of  what  was 
learned  during  the  analyses  performed  above;  professors  identified  changes  that  were 
important.  The  identified  options  can  be  noted  in  the  GAPS  Summary  described  above  (B5) 
or  when  reading  the  Portfolio  summary,  where  professors  use  C to  note  consideration  of 
models,  styles,  etc.  This  program  component  was  also  where  we  began  to  formally  use  the 
“Reversed”  instructional  design.  Professors  were  to  intentionally:  (1)  determine  what  they 
wanted  students  to  be  able  to  know  about  or  do  upon  completing  their  course  and  (2) 
determine  what  evidence  of  learning  was  acceptable  to  confirm  student  learning.  All  other 
instructional  decisions  follow  response  to  these  two  primary  questions  when  using  the 
“reversed  and  intentional”  instructional  design  process  (Dick  & Carey,  1996;  Wiggins  & 
McTighe,  1995,  2005;  Scarborough,  2006). 

This  led  us  to  the  first  stage  of  the  2005  course  redevelopment(s).  From  this  point  forward, 
the  course  will  be  identified  as  the  2006  course.  We  used  a worksheet  that  was  designed  so 
all  ABET(EAC/TAC)/NAIT  Eearning  Outcomes  were  listed  for  the  professors.  Beneath  each 
standard  or  outcome,  the  embedded  general  education  goals  were  also  provided,  and  beside 
that  column.  Bloom’s  Knowledge  Dimensions  were  listed  so  professors  could  determine 
which  dimension  best  identified  the  knowledge  content  of  their  courses.  Also  there  were 
columns  for  identifying  which  of  Bloom’s  Cognitive  Dimensions  each  outcome  would 
achieve.  Einally,  a column  was  included  to  indicate  Dale’s  level  of  active  versus  passive 
learning  that  was  expected  to  occur.  Each  professor  then  redeveloped  the  student  learning 
outcomes.  If  an  outcome  was  complex,  representing  a cluster  of  knowledge  concepts  or 
multiple  knowledge  dimensions,  then  they  were  to  state  these  appropriately  and  break  them 
out,  also  aligning  each  one  beside  the  corresponding  ABET  or  NAIT  standard  or  outcome 
and  showing  the  embedded  general  education  knowledge  expectations.  In  case  an  outcome 
addressed  only  a partial  aspect  of  the  national  outcomes  or  where  there  were  partial  aspects 
of  the  NIU  general  education  goals  expected,  then  they  used  the  highlight  to  identify  them. 
(Note  an  example  on  the  form  below.)  This  program  component  engaged  the  professors  in  a 
way  that  revealed  how  much  they  had  learned.  They  were  using  better  language  for  the 
outcomes,  terminology  that  was  active,  measurable,  and  more  specific;  breaking  the 
outcomes  into  second  and  third  levels  helped  the  professors  to  be  more  specific  - more 
clearly  revealing  what  was  important  to  learn.  It  was  much  easier  to  determine  Bloom’s 
Cognitive  Dimensions  because  the  outcomes  were  written  more  explicitly  and  broken  out  if 
complex  clusters.  This  was  the  first  product  for  their  revised  2006  courses.  What  they  wanted 
students  to  know  about  or  be  able  do  upon  completing  their  course  was  answered.  The  terms 
“intentional  and  reversed”  are  critical  to  us.  We  realize  that  some  professors  have  the  gift  of 
accomplishing  something  good  and  worthwhile  in  their  classrooms  whether  fully  planned  or 
not;  however,  we  are  determined  that  what  is  learned  should  be  intentional,  planned,  and 
formal  in  nature.  This  does  not  dismiss  the  benefit  of  the  informal  learning  or  extensions  that 
normally  occur  beside  the  formal  and  intentional  components.  Completing  this  activity  led  us 
to  the  second  aspect  of  development;  using  the  “reversed”  instructional  design  process. 
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faculty  designed  and  developed  the  student  assessments,  or  the  tools,  processes,  and 
procedures  to  collect  evidence  of  learning. 

The  following  chart  (blank)  presents  ABET  Engineering  outcomes  A,  ...H,I,J,K  just  to 
present  the  worksheet.  The  complete  worksheet  includes  all  Engineering  Outcomes  A-K,  and 
the  embedded  NIU  General  Education  Goals,  Bloom’s  Knowledge  and  Cognitive 
Dimensions,  and  Dale’s  Cone.  A second  version  of  the  form  was  used  for  the 
ABET(EAC/TAC)/NAIT  standards  and  outcomes  for  the  professors  teaching  in  the 
engineering  technology  and  technology  programs.  Selected  outcomes  are  shown  below;  once 
again,  the  professors  used  a form  that  presented  all  outcomes.  Professors  added  their 
individual  course  student  learning  outcomes  aligning  them  with  the  national  outcomes.  The 
stars  identify  levels  of  Bloom’s  accomplished  on  the  Cognitive  Dimension;  checks  (J) 
indicate  the  Knowledge  Dimensions.  (This  provides  a view  of  what  a completed  form 
visually  presents.)  It  is  important  to  remind  ourselves  that  professors  sometimes  do  not 
realize  it  is  not  their  responsibility  to  address  every  national  outcome  in  every  course,  but 
instead  their  courses  contribute  to  the  total  of  all  program  outcomes  required  by  the 
accreditation  agencies. 

Each  version  (ABET  Engineering  and  ABET(EAC/TAC)/NAIT)  of  the  worksheet 
completely  presented  the  national  standards  or  outcomes  with  embedded  NIU  General 
Education  Goals.  Each  professor  completed  one  of  these  worksheets  for  the  2006  course 
he/she  was  redeveloping.  The  form  clearly  presented  what  outcomes  were  and  were  not 
addressed  by  each  course;  it  identified  the  embedded  general  education  knowledge  or 
inherent/embedded  mathematics,  science,  and  communication  expected  of  students.  Then  the 
form  revealed  Bloom’s  Cognitive  and  Knowledge  dimensions  so  professors  could  determine 
where  they  needed  to  increase  learning  to  higher  levels  or  check  to  validate  or  improve  the 
knowledge  content.  The  Knowledge  dimension  assisted  professors  in  examining  their 
knowledge  content  more  deeply  than  topical  outlines  while  also  focusing  on  the  source  of 
knowledge,  and  Dale’s  Cone  reminded  them  to  keep  learning  active  and  of  high  integrity. 
Now  our  professors  understand  that  their  courses  contribute  to  the  overall  program  outcomes 
and  that  they  should  be  clear  about  which  ones  they  attain  in  depth,  and  so  on.  Our  college 
uses  a metric  for  that  purpose,  but  even  so,  we  needed  to  revisit  that  point  so  the  courses 
became  more  focused  and  realistic  about  what  could  be  accomplished  well.  Table  7 presents 
a blank  form  for  the  engineering  program;  Table  8 presents  the  form  for  engineering 
technology  and  technology.  It  is  completed  so  the  reader  can  see  what  occurs.  See  Portfolio 
Section  B7  for  the  complete  example  and  Section  Cl  for  blank  worksheets. 
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Table  A.5.7:  ABET 


engineering  Outcomes  Worksheet 


(See  Portfolio  Cl  for  worksheets) 


ABET  Engineering 
Outcomes 

**Partial  list  of  outcomes. 

Bloom's 

Knowledge 

Dimension 

Dale’s 

Cone 

Passive/ 

Active 

Student 

Learning 

Outcome 

Bloom’s  Cognitive  Process  Dimensions  - Traditional  and  Revised 

Knowledge 

Remember 

Compre- 

hension 

Under- 

stand 

Application 

Apply 

Analysis 

Analyze 

Synthesis 

Evaluate 

Create 

Evaluate 

A.  apply  knowledge  of 
math,  science,  engineering 

NIU  Gen  Ed  Goals  - 
Students: 

a.iii.  perform  basic 
computations,  display 
facility  with  use  of 
formal  and 
quantitative  reasoning 
analysis  and  problem 

■ 

Factual 

Conceptual 

Procedural 

Meta- 

Cognitive 

n.  understand  impact  of 
engineering  solutions  in  a 
global  economic,  environ- 
mental, societal  context 

NIU  Gen  Ed  Goals  - 
Students: 
d.  develop  social 
responsibility  and 
preparation  for 
citizenship  through 
global  awareness, 

pnvirnnmpntnl 

Factual 

Conceptual 

Procedural 

Meta- 

Cognitive 

[.  recognize  the  need  for, 
and  have  capability  to 
engage  in  lifelong  learning 

NIU  Gen  Ed  Goals  - 
Students: 

a.iv.  are  able  to  access 
and  use  various 
information  sources. 
Internet,  text,  field 

Factual 

Conceptual 

Procedural 

Meta- 

Cognitive 

J.  knowledge  of 
contemporary  issues 

NIU  Gen  Ed  Goals  - 
Students: 
d.  develop  social 
responsibility  and 
prepare  for  citizenship 
through  global 
awareness,  environ- 
mental sensitivity,  and 
an  appreciation  of 
cultural  diversity. 

Factual 

Conceptual 

Procedural 

Meta- 

Cognitive 

K.  use  techniques,  skills, 
and  modern  engineering 
tools  necessary  for 
engineering  practice 

Factual 

Conceptual 

Procedural 

Meta- 

Cognitive 

To  present  a completed  section,  below  is  the  merged  ABET/TAC  and  NAIT  worksheet  for  the  Department  of 
Technology’s  Engineering  Technology  and  Technology  programs  (Tech  496).  This  is  a partially  completed 
form. 
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Table  A.5.8:  CE: 


T’s  496  Course  (See  Portfolio  for  complete  form  B7  and  Cl  for  worksheet) 


ABET(EAC/TAC)/N 
AIT  Engineering  & 
Technology 
Outcomes 
**Partial  list  of 
outcomes. 


Bloom's 

Knowledge 

Dimension 

•1  = Yes 


Dale’s 

Cone 

Active 

Partici. 

Passive 


Student 

Learning 

Outcome 


Bloom’s  Knowledge  Dimensions 

Note:  CT  = Critical  Thinking  high  >/=gDDd  !=verygood 


☆ 


:excellent 


Knowledge 

Remember 


Comprehension 

Understand 


Appli* 

cation 

Apply 


Analysis 

Analyze 


Synthesis 

Evaluate 


Create 

Evaluate 


A.  Mastery  of 
knowledge, 
techniques,  skills, 
modern  tools  of 
disciplines. 


ypactual 

yConcep- 

tual 

yprocedural 

yMeta- 

Cognitive 


Highly 

Active 


6.T0 

demonstrate 

effective 

project: 

a.  planning 

b.  initiation 

c.  execution 

d.  termination 


y 


y 


CT 


CT 


CT 


CT 


B.  Apply  current 
knowledge  and  adapt 
to  emerging 
applications  of  math, 
science,  engineering, 
and  technology. 


NIU  Gen  Ed  Goals 
-Students: 
a.iii.  perform  basic 
computations, 
display  facility  with 
use  of  formal  and 
quantitative 
reasoning  analysis 
and  problem 
solving,  and 
interpret 
mathematical 
models  and 
statistical  infor.. 


ypactual 

Knowledge 

yConcep- 

tual 

Knowledge 

yProcedural 

Knowledge 

yMeta- 

Cognitive 

Knowledge 


Highly 

Active 


8.  To  integrate 
mathematics, 
the  sciences, 
communi- 
cation, 

management, 
technical,  and 
technological 
knowledge  and 
skills;  to 
accomplish 
team  and 
project 
objectives. 


y 


y 


CT 


CT 


CT 


CT 


D.  Ability  to  apply 
creativity  in  the 
design  of  systems, 
components,  or 
processes  appropriate 
to  program 
objectives. 


NIU  Gen  Ed  Goals 
- Students: 
c.  develop  an 
understanding  of 
the  relatedness  of 
various  disciplines 
by  integrating 
knowledge  from 
several  disciplines 
and  applying  that 
knowledge  to  an 
understanding  of 
important 
problems,  issues. 


ypactual 

Knowledge 

yConcep- 

tual 

Knowledge 

yProcedural 

Knowledge 

yMeta- 

Cognitive 

Knowledge 


Highly 

Active 


8.  To  integrate 
mathematics, 
the  sciences, 
communica- 
tion, manage- 
ment, 
technical, 
knowledge  & 
project 
objectives 

a.  design  a 
vehicle  to 
technical 
specifications 

b. build  the 
vehicle  to 
techninical 
specifications 
^solve  tech 
problems 
associated  with 
deisgn, 
construction, 
and  evaluation 
d.test/evaluate 
vehicle  against 
tech,  specs. 


y 


y 


CT 


CT 


CT 


CT 


(Scarborough,  2006  incorporating  ABET  and  NAIT  wit 


tiNIU) 


The  following  outline  served  as  a worksheet  to  prepare  student  learning  outcomes  for  the  form  above  and/or 
to  serve  as  a graphic  organizer  or  format  so  professors  could  begin  to  visualize  and  broadly  link  outcomes 
and  assessments.  Some  professors  outlined  their  student  learning  outcomes  before  transferring  them  to  the 
worksheet  above  using  this  form;  others  worked  directly  on  the  worksheet  above.  Either  way  professors 
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began  to  more  deeply  understand  that  everything  must  be  logically  connected,  clearly  linked  and  mapped. 
The  worksheet  revealed  the  broader  link  between  outcomes  and  assessments,  itemizing  outcomes  to 
assessments  (e.g.,  tests  or  performance  tasks)  and  then  to  specific  test  items  or  rubric  criteria. 


Table  A. 5. 9:  Tech  496  Industrial  Project  Management 


Students  will  be  able  to:  Student  Learning  Objectives  and  Outcomes 

Corresponding  Assessments: 

1.  Identify  and  describe  major  problems,  issues,  concerns,  and  solutions  that  relate  to 
projects,  project  management,  project  teams,  and  project  leaders: 

Identify  problems,  issues,  concerns,  and  solutions  (PICS)  that: 

a.  occur  during  projects 

b.  relate  to  project  management 

c.  occur  during  team  engagement  on  projects  between  team  members  and/or  team  leaders 

d.  occur  for  team  leaders  of  projects 

e.  are  specific  to  international  projects 

f.  occur  during  projects  that  are  executed  by  a multi-cultural  teams  and  involve  ethnically  diverse 
team  members  working  together 

g.  are  specific  to  team  leaders  of  international  projects  and/or  multicultural  teams  executing 
projects 

Text  Project 
Literature  Study 
Paper 
Case  Study 
Career  Project 
Team  Project 

2.  Identify  and  describe  best  practices  for  managing  projects  and  leading  teams,  including 
international  projects  and  multi-cultural  teams. 

Literature  Study;  Text  Project 
Case  Study;  Paper,  Team  Project 

3.  Perform  effectively  on  a project  team  (hopefully  multicultural  team)  to  complete  a 
technical  project. 

a.  To  engage  in  conflict  resolution  to  resolve  team  issues. 

b.  To  perform  team  and  peer  assessments  throughout  the  project 

c.  To  execute  a technical  project 

Team  Participation  Assessment 
Team  Assessment 
Professor's  Assessment 
Team  Project  Outcomes 

4.  Prepare  the  team  for  project  work  by: 

a.  developing  a team  operations  manual 

b.  developing  a peer  and  team  assessment  system 

c.  creating  the  team  organization  and  process 

d.  developing  an  individual  project  plan 

e.  participating  in  the  development  of  a team  project  plan 

Team  Operating  Manual 
Peer  Assessment  System 
Use  of  Peer  Assessment  System 
Team  Plan 

5.  Exhibit  leadership  and/or  participation  while  engaged  in  a team  community  service 
project. 

a.  plan  ; b.  initiate  ; c.  execute;  d.  terminate 

Service  Project  Report  & Evaluation 

6.  Demonstrate  effective  project: 

a.  planning;  b.  initiation;  c.  execution;  d.  evaluation;  e.  termination;  f.  problem  solving; 
g.  leadership 

Individual  Project  Plan;  Individual  Portfolio 
Peer  Assessment;  Team  Assessment 
Team  Participation-  Assessment-Project 
feedback 

Team  Project  Plan;  Team  Portfolio 
Team  presentation,  Website 

7.  Demonstrate  effective  use  of  project  management  techniques  and  tools  in  the  management 
of  a technical  project. 

a.  plan;  b.  initiation;  c.  execution;  d.  termination;  e.  evaluation 
f.  MS  Project;  g.  finance  procedures;  h.  SWOT  Analysis; 

i.  procurement  procedures 

j.  scheduling 

k.  MACE  procedures  and  process;  1.  project  evaluation 

Individual  Project  Plan 
MS  Project  Test 
Individual  Portfolio 
Peer  Assessment 
Team  Assessment 
Team  Plan 

Team  presentation,  portfolio,  and  website 

8.  To  integrate  mathematics,  the  sciences,  communication,  management, 
technical,  technological  knowledge  and  skills  to  accomplish  team  and 
project  objectives,  a.  design  a vehicle  to  technical  specifications;  b.  build 
the  vehicle  to  technical  specifications;  c.  solve  technical  problems 
associated  with  design,  construction,  and  evaluation  specifications 

Project  Plan 

Vehicle  Design  Specifications 
Project  Evaluation  Procedures 
Executive  Presentation 
Project/Team  Website 
Team  Portfolio 
Vehicle  Operations  Manual 
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Student  Assessment 
(See  PowerPoint  Presentation  - D4) 


Test  Development 

The  test  development  program  component  was  based  on  the  new  student  learning  outcomes 
for  each  course  and  the  levels  of  Bloom’s  Cognitive  Dimensions  that  each  professor  desired 
to  achieve  with  his/her  students  for  assessments;  therefore,  the  professors  created  a Table  of 
Specifications  based  upon  Bloom’s  Taxonomy  and/or  any  other  specifications  they 
individually  desired.  They  considered  the  validity  of  test  items;  why  to  use  many  discrete, 
objectively  scored  items;  and  how  to  construct  multiple  choice,  items,  short  answer,  and 
true/false  items.  They  began  to  understand  that  the  learning  outcomes  prioritized  as  primary 
were  those  that  also  needed  to  be  more  thoroughly  reflected  in  test  content  and  items,  that  the 
secondary  items  were  to  be  well  represented  on  tests  as  well,  and  that  many  of  the  second  and 
third  level  or  “other”  outcomes  could  not  be  minimally,  if  at  all,  reflected  in  test  content  or 
items.  The  professors  also  learned  to  consider  test  development  from  a time  factor 
perspective,  as  well  as  from  the  perspective  of  what  is  important  to  measure,  and  to  realize 
that  not  everything  can  be  tested  in  a single  test  time-period  of  one-hour  duration.  The 
professors  began  to  understand  it  is  important  to  develop  a test  with  a variety  of  item  types. 
To  reinforce  this  during  the  item  development  process,  they  were  asked  to  create  multiple 
questions  of  different  types  for  each  student  learning  outcome  for  the  entire  course,  thus 
developing  a test  item  bank.  This  would  also  prepare  them  to  be  able  to  retest  something  on 
the  final  exam  that  had  been  on  the  midterm  but  with  a different  question.  A very  important 
discussion  that  occurred  was  about  “easy”  items.  The  professors  revealed  great  concern  about 
making  students  comfortable  by  including  some  intentionally  easy  questions  that  all  students 
would  answer  correctly,  thereby  receiving  those  points.  We  understood  their  motive  and  goal 
but  tried  to  convince  them  that  it  was  better  to  focus  on  the  development  of  well  designed 
and  constructed  items  that  were  clear,  had  integrity,  were  directly  linked  to  what  was  taught, 
and  provided  well  rounded  opportunities  for  students  to  show  evidence  of  learning  (e.g., 
varied  item  types,  etc.).  Once  they  completed  item  development,  they  then  assembled  their 
tests:  one  comprehensive  midterm  examination  and  one  comprehensive  final  examination.  If 
professors  desired  to  have  non-objective,  open-ended  problems,  that  was  fine;  however,  the 
non-objective  questions  were  arranged  on  the  tests  such  that  the  analysis  procedures  were  not 
affected  by  those  items  on  the  answer  sheets.  The  tests  were  reviewed  and  feedback  was 
provided  to  each  professor.  Professors  were  asked  to  revise  their  tests  based  upon  the 
feedback  by  the  program  component  leader.  These  tests  were  then  administered  as  the 
midterm  and  final  exams  for  the  course  during  the  2006  experimental  research  semester  that 
followed.  Each  professor  then  engaged  in  test  analysis  to  determine  the  strengths  and 
weaknesses  of  the  tests,  which  reinforced  the  actual  on-going  use  of  test  analysis  for 
diagnostic  purposes.  They  prepared  an  informal  diagnostic  write  up  describing  how  the 
analyses  were  to  be  used  diagnostically  to  improve  tests  and  instruction  in  the  future. 


3 

Recall  that  the  professors  learned  how  to  perform  test  analyses  on  their  2005  tests  during  the  test  analysis 
program  component  earlier  (e.g.,  item  analysis,  item  statistics,  item  difficulty  and  discrimination)  using  a Table 
of  Specifications,  etc. 
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It  might  be  important  to  understand  several  points  of  diseussion.  We  diseussed  validity  and 
reliability  but  did  not  go  deeper  into  those  topics.  We  also  discussed  how  to  externally 
validate  the  tests  using  external  field-based  engineers  and  technologists,  a process  that  can 
validate  the  content  and  its  application.  Also  we  discussed  using  peers  to  validate  that  the 
tests  are  measuring  what  is  intended.  Finally,  one  of  the  most  important  considerations  - how 
does  one  interpret  what  a student  knows  or  can  actually  “do”  using  tests.  We  discussed  the 
philosophy  that  “typical”  tests  only  reveal  what  students  might  know  about,  not  what  they 
can  actually  do  (Linn  & Baker,  1996  as  cited  in  National  Society  for  the  Study  of  Education  - 
NSSE).  Test  results,  unless  developed  well  to  the  point  of  requiring  higher  cognitive  levels  of 
performance,  serve  only  as  indicators  of  what  students  might  be  able  to  do  or  perform  - 
valued  real  world  performances.  We  understand  that  if  the  test  items  are  written  to  achieve 
the  higher  levels  of  Bloom’s  cognitive  process  dimensions,  they  can  in  fact  reflect  what 
students  can  do,  e.g.,  problem  solving.  We  used  the  “tests  are  usually  useful  only”  as 
“indicators  of  valued  ‘real-world’  performances”  as  stimulus  to  segue  into  the  need  to 
include  performance  assessments  to  measure  student  learning  (p.  85).  Eurthermore,  we 
established  that  the  professors  had  very  limited  assessment  plans  for  their  courses.  Although 
there  were  some  projects,  these  were  not  formally  structured  and  had  no  formal  grading 
criteria  or  rubrics.  Both  students  and  professors  are  at  a distinct  disadvantage  when  there  is 
little  to  no  structure  or  few  to  no  grading  criteria  to  make  clear  what  is  expected  on 
performances,  how  assessments  will  be  graded  and  counted  in  the  overall  course,  and  to 
prevent  ambiguous  and  inconsistent  grading  by  the  professor.  Therefore,  it  quickly  became 
apparent  that  a more  multifaceted  and  balanced  assessment  plan  was  needed  for  each  course, 
where  formal  performance  tasks  and  rubrics  would  benefit  the  students  and  course 
knowledge  attainment  as  well  as  provide  a greater  range  of  tools  or  procedures  with  which 
students  could  provide  evidence  of  learning.  Therefore,  we  considered  tests  at  the  lower 
levels  of  Bloom’s  cognitive  dimensions  to  serve  only  as  indicators  of  what  students  can 
actually  do;  of  course,  tests  that  do  achieve  upper  levels  of  Bloom’s  may  reveal  more.  This 
led  us  to  consider  how  to  broaden  the  assessment  plans  for  each  course  to  include 
performance  tasks  and  rubrics  for  projects  or  performances  that  would  extend  what  could  be 
determined  about  student  attainment  of  knowledge  in  the  courses.  We  also  encouraged 
professors  to  expand  their  assessment  plans  beyond  tests  and  performance  tasks  so  students 
would  experience  even  more  and  varied  opportunities  to  provide  evidence  of  what  they 
learned.  The  feedback  from  the  professors  on  the  value  of  learning  how  to  analyze  and 
develop  tests  was  extremely  positive.  (Kuhs  et  ah,  2001;  Nitko,  2004).  (See  Tests  Section  - 
B6) 

Outcomes  by  Test  and  Items  Analysis  (See  Sections  B8  and  Cl) 

Once  the  2006  tests  were  developed,  we  engaged  the  professors  in  another  analysis  using  a 
worksheet  to  stimulate  and  reinforce  the  “Intentional”  and  “Reversed”  instructional  design 
process.  As  described  above,  professors  were  first  asked  to  determine  what  they  wanted 
students  to  be  able  to  know  about  or  do  upon  leaving  their  course  and  then  what  evidence  of 
learning  would  be  acceptable.  We  required  the  professors  to  develop  objective  midterm  and 
final  exams,  even  if  they  preferred  problem  solving  and  non-objective  items,  because  we  felt 
they  needed  to  learn  how  to  develop  tests  with  higher  integrity.  We  also  required  them  to 
design  and  develop  three  complex  performance  tasks  with  clusters  of  performances 
embedded  within  the  “real-world”  tasks  and  the  corresponding  rubrics  with  standards  of 


35 


performance  and  criteria  for  each.  We  asked  them  to  overlap  or  connect,  both  the  midterm 
and  final  exam,  to  a performance  task.  Finally,  we  asked  them  to  develop  a third  performance 
task  and  use  it  anywhere  within  the  course  content  or  schedule.  Our  purpose  in  doing  that 
was  to  have  them  realize  that  knowledge  varies  and  how  it  can  best  be  measured  also  varies. 
Some  knowledge  can  be  measured  well  by  both  tests  and  performances;  some  to  traditional 
tests;  while  other  knowledge  is  best  measured  by  performances.  Also  some  content  can  be 
measured  more  deeply  if  first  tested,  perhaps  somewhat  superficially,  and  then  through 
performances.  Overlapping  testing  and  performance  assessment  is  sometimes  beneficial.  But 
clearly  the  professors  needed  to  consider  the  relationship  between  traditional  tests  and 
performances  and  what  they  could  accomplish  as  measurement  tools  for  determining  what 
students  learned  and  to  what  level  they  have  learned  the  knowledge  or  skill.  Finally,  it  was 
important  for  the  professors  to  realize  that  students  vary  in  their  ability  to  reveal  what  they 
have  learned  and  need  a variety  of  opportunities  to  best  provide  evidence  of  learning.  To  us, 
the  best  courses  are  those  with  many  and  varied  types  of  opportunities  for  students  to  show 
the  professor  what  they  have  learned.  Most  professors  in  our  program  learned  that  they  were 
too  limited  in  the  breadth,  depth,  and  number  of  assessments  required  throughout  the  course 
and  that  their  assessments  were  not  integrated  into  the  learning  process  but  were  instead 
artificially  separated.  We  prefer  “assessment  as  learning,”  where  students  self-monitor  and 
self-correct  or  make  adjustments  while  learning,  and  providing  evidence  of  learning.  We 
realize  that  assessment  of  learning,  where  the  teacher  or  other  students  are  the  reference 
points  or  assessors,  and  that  assessment  for  learning  is  critical  for  measuring  achievement  of 
the  external  standards  of  our  accrediting  agencies  are  all  important.  However,  we  advocate 
that  “assessment  as  learning”  is  critical  and  most  beneficial  for  students  while  enrolled  in  our 
courses,  and  especially  toward  life-long  learning  (Marzano  et  ah,  1993;  Scarborough,  2004, 
Chapter  12;  Wiggins,  1998). 

This  is  where  we  began  to  more  deeply  consider  the  relationship  between  the 
ABET(EAC/TAC)/NAIT  standards  or  outcomes  and  student  learning  assessments.  We 
wanted  to  see  a much  more  direct  link  between  the  national  standards,  the  student  learning 
outcomes  for  each  course,  and  the  assessment  tools  for  each  course.  Using  the  simple 
worksheet  below,  professors  once  again  mapped  student  learning  outcomes  to  tests  and  test 
items,  to  performance  tasks  and  rubrics,  and  to  each  criterion.  In  completing  this  map,  any 
gaps  or  missing  links  between  what  knowledge  professors  desired  to  measure  and  what  they 
were  actually  including  for  measurement  on  the  assessments  became  obvious.  This  is  a 
critical  analysis  in  developing  any  course,  as  it  reveals  important  information  about  the 
quality  of  connections  between  what  should  be  taught  (as  identified  by  the  outcomes),  and 
what  learning  is  being  measured  (as  identified  by  the  test  items  or  rubric  criteria).  The 
process  leads  to  improved  assessments  or  changes  in  outcomes  (Einn  & Baker,  1996  as  cited 
in  NSSE,  1996;  Nitko,  2004)  (To  learn  more  about  Performance  Assessment,  see  Sections 
B9,C.3,  andD4) 


36 


Table  A. 5. 10:  Tech  496  Industrial  Project  Management  (See  Sections  B8  and  Cl) 


Student  Learning  Objectives  and  Outcomes 

Assessments: 

Test  & Performance  Task  Alignments 
Midterm  & Final 

Student  Learning  Outcomes-Primary 

Student  Learning 
Outcomes  - 
Secondary 

Corresponding 
Test  and 
Tests  Items 

Corresponding 
Performance 
Tasks  and  Rubrics 

1 

a 

b 

c 

2 

a 

b 

c 

To  learn  about  performance  assessment,  we  engaged  the  professors  in  the  development  of 
performance  tasks  and  corresponding  rubrics.  Thus,  the  learning  and  professional  growth  on 
the  performance  assessment  knowledge  component  of  the  program  was  taught  and  measured 
by  the  professors’  performance  on  the  task  of  designing  and  developing  three  complex 
performance  tasks  and  three  corresponding  rubrics  for  scoring  task  achievement.  Using  the 
rubrics  below  as  guiding  criteria,  they  each  designed  three  complex  performance  tasks  and 
corresponding  rubrics.  The  performance  assessments  were  added  to  their  courses  as  a new 
assessment  strategy  and  procedure. 

It  is  important  to  note  that  one  performance  task/rubric  was  designed  to  correspond  with  the 
midterm  and  another  with  the  final  exam,  using  the  logic  that  objective  tests  usually  reflect 
what  students  know  or  know  about  rather  than  what  they  can  do.  Therefore,  we  used  an 
unusual  scenario  where  the  professors  “linked”  the  objective  midterm  exam  to  a midterm 
performance  task/rubric  and  an  objective  final  exam  to  a final  performance  task/rubric.  They 
also  developed  a third  performance  task/rubric  and  choose  how  and  when  to  use  it  during  the 
course.  They  were  asked  to  “match”  where  they  thought  the  test  items  and  performance  tasks 
“overlapped”  or  measured  the  same  or  similar  content.  An  assumption  was  made  from 
studying  the  literature  that  performance  assessment  measures  different  aspects  of  learning, 
sometimes  deeper  levels  of  learning  through  use  of  knowledge  in  more  active  or  engaging 
ways,  problems,  projects,  etc.  But  performance  assessment  can  also  measure  some  of  the 
same  aspects  of  learning  as  objective  tests.  Also  some  of  the  professors  designed  their  tests  to 
incorporate  some  level  of  performance  in  subjective  or  problem-based  items.  In  examining 
and  analyzing  the  tests,  the  objective  items  were  separated  from  the  more  performance-based 
items. 

Professors  were  provided  a presentation  about  performance  assessment.  Performance  tasks 
and  rubrics  were  discussed,  and  they  received  many  examples  of  tasks  and  rubrics.  They  also 
received  books  on  the  topic  as  part  of  their  new  library  on  teaching  and  learning.  Their 
performance  tasks  and  rubrics  reflect  the  ABET  or  NAIT  outcomes.  One  professor  had  used 
rubrics  before,  “less  formalized,  somewhat  more  of  a check  off  oriented  form  with  less  well 
developed  descriptors  or  criteria  and  levels  of  achievement,”  but  none  of  the  professors  had 
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developed  or  used  formal,  written,  real-world,  scenario-based  performance  tasks  with 
corresponding  and  formal  rubrics  before  this  initiative.  The  one  who  had  used  a rubric, 
revised  it  after  the  program,  making  it  more  formal  with  clear  and  distinctive  performance 
standards  and  criteria  for  each  performance  level.  Thus,  there  were  no  previous  instruments 
to  review  from  the  baseline  semester.  Fall  2005,  to  compare  to  these.  Therefore,  we  judged 
them  based  upon  the  rubrics  below  (Linn  & Baker,  1996  as  cited  in  NSSE,  1996;  see  also 
Angelo  & Cross,  1993  Chatterji,  2003;  Nitko,  2004;  Walvoord  & Anderson,  1998;  Wiggins, 
1998). 

Faculty  Development  Program  - Performance  Task:  Design  and  develop  three  complex,  real 
world  and  authentic  performance  tasks  with  corresponding  rubrics.  The  professors  based 
each  of  the  performance  tasks  upon  selected  student  learning  outcomes,  reflecting  the  ABET 
or  NAIT  outcomes  and  authentic  real  world  tasks  relevant  to  learning  outcomes,  and  used 
rubrics  for  student  assessment  during  the  experimental  research  course,  Eall  2006.  See  the 
rubrics  below  for  the  achievement  standards  and  criteria. 

Professors’  Performance:  The  professors  accomplished  the  performance  task  well.  The 
process  involved  drafting  initial  and  authentic  real  world  scenarios  with  embedded  task 
clusters  and  a corresponding  rubric  instrument  for  each  task.  The  program  leader  provided 
feedback  one-on-one.  The  professors  shared  their  drafts  with  each  other  and  benefited  from 
the  group  critique  process.  The  group  process  worked  especially  well.  The  tasks  and  rubrics 
were  finalized;  the  program  leader  approved  them;  and  then,  each  professor  used  the  tasks 
and  corresponding  rubrics  successfully  with  students  during  the  2006  experimental  research 
semester.  As  with  test  analysis  and  development,  the  feedback  from  the  professors  on  the 
value  of  learning  to  design,  develop,  and  use  performance  tasks/rubrics  was  extremely 
positive. 

The  following  rubrics  were  used  to  guide  the  professors  in  the  development  of  the  three 
performance  tasks  and  corresponding  rubrics  for  each  task.  (See  Section  B8) 
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Table  A.5.11;  Rubric  for  Assessing  the  Quality  of  a Performance  Task 

Key  Components  - Properly  Designed  Performance  Tasks  must 

I.  Be  based  on  content  standards  established  by  ABET  or  NAIT 

II.  Describe  a “real-life”  scenario;  are  real  world,  authentic  tasks;  require  active  performances 

III.  Involve  students  in  complex  reasoning  - critical  thinking  at  upper  levels  of  Bloom’s  Cognitive  Dimension 

IV.  Require  students  to  collect  and  process  information,  using  it  for  an  authentic  purpose 

V.  Incorporate  “habits  of  mind” 

VI.  Require  student  collaboration  and  cooperation;  incorporates  “individual”  and  “group”  learning  and  performance 
accountability 

VII.  Result  in  a tangible  product  and/or  communication  activity 

For  each  component,  there  are  descriptors  reflecting  levels  of  achievement  possible: 

I.  The  Performance  Task  is  based  on  the  ABET  or  NAIT  standards 

a.  The  Performance  Task  is  directly  related  to  the  ABET  or  NAIT  standards. 

b.  Learning  standards  are  apparent,  but  the  relation  to  the  task  and/or  national  standards  is  sketchy  or  not  apparent. 

c.  The  Performance  Task  does  not  appear  to  be  based  on  the  standards/outcomes,  course  or  national. 

II.  The  Performance  Task  describes  a “real-life”  scenario  that  is  authentic  and  requires  active  performance. 

a.  The  scenario  described  in  the  task  accurately  mirrors  an  activity  in  the  community  of  practice  outside  the 
classroom. 

b.  The  scenario  described  in  the  task  simulates  an  activity  in  the  community  of  practice  outside  the  classroom. 

c.  The  scenario  described  in  the  task  contains  some  aspects  of  activity  outside  the  classroom  but  is  largely  contrived. 

d.  The  scenario  described  in  the  task  is  an  academic  exercise  that  usually  takes  place  only  in  the  context  of  an 

academic  setting. 

III.  The  Performance  Task  involves  students  in  complex  reasoning-critical  thinking  processes  at  upper  levels  of 

Bloom’s  Cognitive  Dimension. 

a.  The  task  requires  students  to  utilize  complex  reasoning  - critical  thinking  skills,  such  as  induction/deduction, 
diagnosis,  abstracting,  experimental  inquiry,  problem  solving;  evaluation,  creation,  synthesis,  etc. 

b.  The  task  requires  students  to  utilize  complex  reasoning  components,  such  as  comparing,  classifying,  decision 
making,  or  investigation. 

c.  The  task  requires  students  only  to  recall  facts. 

IV.  The  Performance  Task  requires  students  to  collect  and  process  information,  using  it  for  an  authentic  purpose. 

a.  The  task  incorporates  a variety  of  information  gathering  techniques  and  information  resources.  Students  are 
required  to  interpret  and  synthesize  information  and  accurately  assess  the  value  of  information  gathered.  They  are 
required  to  collect  the  right  information  for  an  authentic  purpose,  e.g.  solve  a problem,  apply  or  use  in  a complex 
project,  etc. 

b.  The  task  requires  students  to  gather  and  synthesize  information,  but  the  value  of  the  information  gathered  is  not 
assessed.  Information  may  not  be  used  for  a purpose. 

c.  The  task  requires  the  students  to  gather  information,  but  not  to  interpret  it. 

d.  The  task  requires  no  gathering  or  processing  of  information. 

V.  The  Performance  Task  incorporates  “Habits  of  Mind.” 

a.  The  task  requires  students  to  make  effective  plans,  use  necessary  resources,  evaluate  effectiveness  of  their  own 
actions,  seek  accuracy,  and  engage  in  activities  when  answers  or  solutions  are  not  immediately  apparent. 

b.  The  task  only  requires  students  to  effectively  plan  or  use  resources. 

c.  The  task  does  not  require  students  to  engage  in  self-regulation,  critical,  or  creative  thinking. 

VI.  The  Performance  Task  requires  student  collaboration  and  cooperation;  incorporates  “individual”  and  “group” 

learning  and  performance  accountability. 

a.  The  task  requires  students  to  use  interpersonal  skills,  work  toward  the  achievement  of  team  goals,  and  perform  a 
variety  of  roles  within  the  team.  There  is  a formal  team  structure  and  process. 

b.  The  task  requires  students  to  work  together  in  teams  but  there  are  no  measures  described  that  ensure  collaboration 
or  cooperation  among  team  members. 

c.  The  task  is  completed  largely  by  students  on  an  individual  basis  rather  than  in  student  teams. 

VII.  The  Performance  Task  results  in  a tangible  product  and/or  communication  activity. 

a.  The  task  result  is  a tangible  product  or  communication  activity  comparable  to  that  commonly  produced  in  business 
or  industry  community  of  practice. 

b.  The  task  results  in  a product  that  is  similar  to  those  completed  in  business  or  industry  community  of  practice,  but 
lacks  several  components  that  make  the  product  realistic. 

c.  The  task  does  not  result  in  a product  or  communication  activity  relevant  to  a business  or  industry  community  of 

practice.  (Scarborough,  2006  [Based  upon  White  & Scarborough,  2004]) 
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Table  A. 5. 12:  Rubric  for  Assessing  the  Quality  of  a Rubric 

Properly  Designed  Rubrics  Must 

I.  Contain  a set  of  key  components/standards  to  be  assessed  that  reflect  the 
student  learning  outcomes  for  the  course,  which  are  directly  linked  to  the 
national  outcomes. 

II.  Include  descriptors  for  each  component/standard  that  are  measurable. 

III.  Have  descriptors-criteria  that  are  indicative  of  observable  student 
performances  or  behaviors. 

IV.  Incorporate  a clear  and  well-defined  scoring  system 

V.  (Optional)  Include  appropriate  weights  for  each  component  and  descriptor 

For  each  component,  there  are  descriptors  reflecting  levels  of  achievement  possible: 

I.  The  rubric  contains  a set  of  key  components  (standards)  to  be  assessed. 

a.  A complete  list  of  key  components-standards  is  provided  for  the  performance  task, 
including  the  embedded  subtasks,  if  a cluster.  The  task(s)  are  directly  connected  to 
student  learning  outcomes  for  course  and  the  national  outcomes. 

b.  Key  components/standards  listed  are  not  exhaustive  for  the  performance  task  and/or 
subtasks  embedded  are  not  clear  enough  for  student  response  or  action;  components  or 
standards  are  not  clearly  connected  to  student  learning  outcomes  for  course. 

c.  Not  all  key  components/standards  describe  student  outcomes;  some  are  not  directly  linked  to 
national  outcomes. 

d.  No  key  components  are  listed. 

II.  The  rubric  includes  a set  of  descriptors-criteria  for  each  key  component  or  standard. 

a.  Descriptors-criteria  for  each  component  or  standard  are  arranged  in  a clear  hierarchy  from 
non-achievement  to  full-achievement. 

b.  Descriptors-criteria  are  present  for  each  component/standard,  but  obvious  levels  in  some  are 
missing. 

c.  Each  component  does  not  have  an  associated  set  of  descriptors-criteria. 

III.  The  rubric  descriptors/criteria  are  clear  and  contain  observable  or  measurable  student 
performances  or  behaviors. 

a.  All  descriptors-criteria  clearly  delineate  levels  of  observable  student  performances  or 
behaviors. 

b.  Most  descriptors-criteria  clearly  delineate  levels  of  observable  student  performances  or 
behaviors. 

c.  Only  a few  descriptors-criteria  clearly  define  levels  of  observable  student  performances  or 
behaviors. 

d.  Descriptors-criteria  do  not  describe  observable  student  performances  or  behaviors. 

IV.  Incorporate  a clear  and  well-defined  scoring  system 

a.  There  is  a well  defined  and  clear  system  for  scoring  each  component-standard  and  its 
descriptors-criteria.  Points  or  percentages  are  assigned  appropriate  to  instructional  emphasis 
and  performance  values. 

b.  The  scoring  system  lacks  definition,  clarity,  and  although  there  is  a scoring  system,  some 
aspects  are  ambiguous,  subjective  or  unclear. 

c.  There  is  no  scoring  system. 

V.  Optional:  Appropriate  weights  are  assigned  to  components  and  descriptors. 

a.  Component-standards  and  descriptors-criteria  are  each  properly  weighted  according  to 

instructional  emphasis  and  performance  values. 

b.  Weights  are  assigned,  but  point  values  do  not  reflect  proper  instructional  emphasis  or 
performance  values  in  all  cases. 

c.  Weights  are  assigned  to  some  performance  standards  and  descriptors,  but  not  others. 
(Scarborough,  2006  [Based  upon  White  & Scarborough,  2004]) 
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Midterm  and  Final  Exam  Performance  Assessment  Correlation 
Further  Thought  About  Performance  Assessment  and  its  Relationship  to  Tests 


Typically,  traditional  objective  tests  are  only  indicators  of  what  students  can  do  with  the 
knowledge  being  measured.  Performing  well  on  a traditional  test  should  not  lead  to  a 
conclusion  about  what  a student  can  do  with  that  knowledge  (e.g.,  how  well  they  can  use  the 
knowledge).  Traditional  or  objective  tests  usually  measure  what  students  know  or  know 
about,  while  performance  assessments  should  engage  students  in  authentic  and  real  world 
performance  tasks  in  which  students  actually  do  something  with  the  knowledge  learned.  It  is 
sometimes  perceived  by  performance  assessment  advocates  that  performance  assessments  (if 
designed,  developed  and  constructed  well)  are  better  evidence  of  what  students  are  capable  of 
doing  with  gained  knowledge.  That  is  assuming  most  traditional  tests  are  written  to  measure 
memory  for  information,  concepts,  theories,  facts.  If,  however,  tests  have  been  written  to 
include  items  that  are  higher  on  Bloom’s  Taxonomy  and  require  more  critical  thinking  or 
problem  solving,  then  those  tests  could  provide  evidence  of  learning  beyond  what  students 
know  about  - what  they  can  “do.”  If  the  problems  are  complex  and  well  constructed,  use  of 
the  knowledge  will  provide  evidence  of  “doing”  or  using  the  knowledge  in  some  way. 

Some  professors  prefer  to  use  tests  intentionally  as  indicators  of  what  students  know  about 
and  then  follow  those  tests  with  performance  tasks  requiring  students  to  solve  problems  or 
engage  in  projects  that  require  higher  levels  of  critical  thinking  (the  manipulation  of  facts, 
theories,  concepts,  information)  in  a context  where  particular  constraints  and  conditions  as 
well  as  tools,  materials,  procedures,  etc.  are  set.  If  this  is  the  goal,  then  a test  and 
performance  task(s)  may  be  designed  to  measure  some  of  the  same  knowledge  while  also 
measuring  different  knowledge,  as  they  are  distinctly  different  types  of  measures  with  the 
potential  to  accomplish  different,  as  well  as  some  of  the  same,  measurement  goals.  Or 
performance  tasks  can  be  used  to  better  understand  the  depth  of  a student’s  learning. 
Therefore,  we  asked  the  professors  to  design  and  develop  objective  midterm  and  final 
examinations  as  well  as  corresponding  performance  task(s)  and  scoring  rubrics  matching  the 
content  where  possible  or  desirable.  They  were  asked  to  identify  the  objective  test  items  that 
they  felt  were  being  measured  on  the  corresponding  performance  tasks,  also  providing  the 
link  to  the  specific  rubric  standards  and  criteria.  This  led  to  an  understanding  of  what  content 
was  not  measured  across  both  types  of  assessments  or  remained  measured  using  one  tool  or 
procedure.  Clearly,  the  professors  began  to  realize  what  they  were  measuring,  what  was  not 
being  measured,  and  also  IF  the  measures  truly  focused  on  the  knowledge  being  taught  and 
outcomes  to  be  achieved  by  the  course. 

A statistical  correlation  was  run  between  the  midterm  exam  and  corresponding  performance 
assessment  and  the  final  exam  and  corresponding  performance  examination  for  each 
professor’s  students.  The  results  led  the  professors  to  consider  the  following: 

1 . Do  they  really  feel  that  there  is  a segment  of  the  objective  tests  and  the 
performance  tasks  where  there  is  a content  match?  If  so,  in  our  program,  no 
external  content  validation  was  required.  We  assumed  the  professors  knew  their 
content  (disciplinary  bodies  of  knowledge).  However,  it  is  important  to  note  that 
professors  should  validate  the  knowledge  content  for  both  the  course  and 
assessments,  as  well  as  the  assessment  procedures,  etc.  externally  in  the  purest 
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sense  of  measurement  or  student  assessment.  That,  however,  takes  more  time  to 
execute  with  a faculty  learning  community  and,  in  our  opinion,  would  be  part  of  a 
Stage  II  faculty  development  program.  Our  focus  was  on  test  analysis,  the 
development  of  objective  tests  with  higher  integrity,  as  well  as  improved  and 
higher  level  test  items,  including  problem  solving  items.  In  addition,  our  program 
focused  on  introducing  them  to  the  design,  development,  and  use  of  performance 
tasks  and  rubrics  as  another  type  of  learning  measurement  procedure  or  tool  to 
broaden  and  better  balance  the  overall  student  assessment  plan  for  the  course. 

2.  How  are  professors  using  the  tests  and  performance  task(s)?  In  our  case,  we 
encouraged  them  to  design  new  tests  with  more  items,  a wider  range  of  item 
types,  items  that  offer  the  opportunity  to  perform  at  various  levels  of  Bloom’s 
learning  (e.g.,  memory  to  synthesis,  even  creativity).  We  then  asked  them  to 
design  and  develop  corresponding  performance  tasks  and  rubrics  to  provide 
students  the  opportunity  to  provide  evidence  of  learning  through  real  world  and 
authentic  performances  by  incorporating  formal  performance  assessments.  So  - 

a.  Do  professors  feel  that  the  objective  tests  are  indicators  of  what  students 
know  and  the  performance  tasks  take  the  students  to  the  next  level  where 
they  are  positioned  to  more  deeply  or  critically  use  the  knowledge 
measured  on  the  objective  tests? 

b.  Do  professors  feel  that  they  can  better  measure  some  types  of  knowledge 
with  objective  tests  and  other  types  of  knowledge  through  performances? 

c.  Other  considerations 

3.  What  might  the  correlation  scores  mean ?^  How  can  they  be  used? 

a.  The  correlation  scores  might  have  little  or  no  meaning. 

b.  The  scores  might  provide  insight  about  students. 

c.  The  scores  might  stimulate  diagnostic  thoughts  about  student  assessment. 

d.  The  scores  might  show  a statistical  relationship  between  tests  and 
performances  if  designed  with  that  goal  in  mind. 

e.  Other 

Although  we  had  hoped  to  use  the  correlations,  we  quickly  realized  that  we  were  not  at  a 
point  in  student  assessment  program  component  to  use  the  correlations  meaningfully.  Nor 
were  the  professors  prepared  to  research  such  a complex  topic  about  which  they  have  just 
begun  to  become  informed.  Therefore,  the  study  of  the  relationship  between  tests  and 
performance  assessment  is  now  on  the  professors’  future  research  agenda. 

Multi-faceted  and  Balanced  Assessment  Plans 

Regarding  student  assessment,  we  began  where  the  professors  were  with  a variety  of  tests 
and  worked  with  them  to  perform  a test  analysis  on  the  tests  used  in  the  2005  course.  Once 
they  completed  the  analysis,  they  each  developed  two  new  tests:  a midterm  and  final  exam. 


4 

Correlations  have  been  computed  in  two  ways:  1)  leaving  zero  scores  in  as  zeros  and  2)  replacing  zero  scores 
with  blanks  or  taking  them  out  (e.g.,  student  was  absent).  Zeros  have  different  meanings  across  professors.  A 
zero  can  be  the  actual  score  for  the  student  or  could  result  from  not  taking  the  test.  Rather  than  make  individual 
determinations,  both  were  computed. 
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for  the  re-developed  course  to  be  taught  during  the  research  semester,  Fall  2006.  Once  that 
was  accomplished,  the  professors  were  introduced  to  performance  assessment  and  rubrics. 

We  wanted  them  to  broaden  the  types  of  performance  assessments,  but  especially  to  add 
formal  performance  assessments.  Therefore,  we  asked  each  of  them  to  develop  three  complex 
performance  tasks  with  corresponding  rubrics  to  help  them  realize  they  could  broaden  their 
types  of  assessment  while  also  realizing  that  this  would  broaden  the  opportunities  for 
students  to  provide  evidence  of  learning.  Many  of  the  professors  had  performances  built  into 
their  2005  tests,  but  the  structure  and  criteria  were  often  weak  and  the  grading  was  far  too 
subjective.  Each  professor  completed  the  development  of  the  performance  tasks  and  rubrics, 
which  turned  out  well  for  the  professors  and  the  students  during  the  research  semester. 

The  Ultimate  Goal-Balanced  and  Multifaceted  Assessment  Plans 

Finally,  the  ultimate  goal  was  to  have  the  professors  realize  there  were  still  many  more  types 
of  student  assessments  to  be  considered,  and  that  the  more  opportunities  for  assessment  and 
the  more  varied  the  types  of  assessments  for  student  assessment,  the  better  the  assessment 
plan  would  be,  as  it  would  become  more  “balanced.”  (A  balanced  student  assessment  plan 
provides  more  reliable  evidence  of  learning,  making  the  determination  of  end-of-course 
grades  less  difficult  and  more  accurately  reflective  of  what  a student  knows  and  can  do.)  An 
old  adage  was  mentioned:  final  exams  should  not  “make  or  break”  the  grade,  but  finals  (and 
midterms  to  that  point  in  the  course)  really  should  confirm  a final  grade  in  the  course. 
Multiple  assessments  of  different  varieties  can  reveal  more  clearly  and  deeply  what  students 
have  really  learned  rather  than  what  they  have  memorized  for  the  short  term.  Therefore,  we 
introduced  the  following  graphic  organizers  as  visuals  for  the  professors  to  consider.  With 
the  visuals  as  an  aid,  they  began  to  understand  that  assessments  should  be  often,  varied,  and 
come  in  all  forms  of  activities  and  that  homework,  papers,  reports,  projects,  case  studies,  and 
so  much  more  also  provide  evidence  of  learning  and,  therefore,  can  be  considered 
assessments  (Kuhs  et  ah,  2001;  Wiggins,  1998).  Figures  8B1  and  8B2  reflect  the  difference 
between  an  unbalanced  assessment  plan  and  the  exemplary  assessment  plan. 
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Figure  A. 5. 8:  Typical  Imbalanced  Assessment 


Authentic  Performance 


(Wiggins,  1998,  p.  115) 


Figure  A. 5. 9:  Exemplary  Assessment  Balance 
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(Wiggins,  1998,  p.  116;  Scarborough  adjustment) 
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Using  the  basic  Kuhs  et  al.  (2001)  model  immediately  below,  each  professor  mapped  his/her 
assessment  plan,  producing  a visual  chart,  like  the  Kuhs-Scarborough  one  (also  below).  The 
chart  served  as  a tool  to  assist  them  in  considering  where  they  could  expand  to  include 
additional  types  of  assessments. 

Figure  A.5.10:  Kuhs  et  al.  Model 


(Kuhs  et  al.,  2001,  p.  157) 
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Mapping  the  assessments  assisted  the  professors  to  more  clearly  realize  if  the  assessments 
were  balanced  regarding  variety  and  to  determine  if  there  were  enough  assessment 
opportunities. 


After  mapping  the  assessments  using  the  Kuhs  model,  above,  the  professors  completed  the 
following  chart  using  percentages  to  reveal  the  assessment  content  by  Bloom’s  Cognitive 
Dimension  levels  for  both  tests  and  performance  tasks.  Some  professors  provided 
information  on  other  types  of  assessments  as  well.  This  helped  them  to  determine  whether 
the  assessments  were  designed  to  engage  students  at  the  higher  level  of  cognitive  processing 
and  critical  thinking.  Also  both  the  map  above  and  the  chart  below  reveal  the  entire 
assessment  picture  for  each  course.  Each  professor  can  clearly  see  if  he/she  has  designed  and 
developed  assessments  to  achieve  the  assessment  goals  for  their  course. 


46 


Table  A. 5. 13:  Bloom’s  Taxonomy  - Cognitive  Dimension  Analysis  Chart  for  Assessments 


(percentage  of  each  assessment  at  each  Cognitive  Dimension) 


Assessment  Type 

Knowledge 

Comprehension 

Appiication 

Analyze 

Synthesis 

Evaluate 

Remember 

Understand 

Apply 

Analyze 

Evaluate 

Create 

Midterm  Exam 

Final  Exam 

Performance  Task  1 

Performance  Task  2 

Performance  Task  3 

Other  Assessments 

The  Student  Assessment  Component  of  the  Program  went  into  much  more  depth  about 
assessment.  Generally,  the  program  content  for  that  component  followed  the  book  chapter 
(Scarborough,  2004)  that  summarizes  the  aspects  of  assessment  important  to  this  project  as 
well  as  identifies  the  critical  sources  for  its  content.  (See  Section  A7) 

The  following  topics  and  related  information  were  presented  and  then  discussed  with  the 
professors  to  broaden  their  perspectives  and  deepen  their  understanding  about  assessment 
practices: 

• Aligning  Achievement  Targets  (ABET/TAC/NAIT  outcomes)  with 
Assessment  Methods 

• Assessment  Quality 

• Balanced  Assessment 

• Performance  Assessment,  Authentic  Assessment,  and  Quality  Rubrics 

• Difference  Between  Tests  and  Authentic  Tasks 

• Learning  Styles  and  Assessment 

• Assessment  of,  for,  and  as  Learning 

Grading 

“Laculty  were  astounded  by  students’  lack  of  knowledge”  (Mitchell,  1998,  p.  128).  More 
important  was  the  widening  gap  in  the  ways  in  which  student  learn  and  the  ways  faculty 
teach!  However,  A’s  everywhere  were  more  abundant  than  ever  at  the  time  of  his  article. 

This  could  suggest  grade  inflation.  Mitchell’s  dean  suggested  that  the  group  of  faculty  get 
together  and  consider  expectations  (for  grades)  as  a group.  There  were,  of  course,  defenders 
of  grade  inflation:  students  are  stronger  academically  now,  lower  grades  will  hurt  their  job 
opportunities,  and  so  many  more.  One  of  the  more  historical  impacts  on  grade  inflation  was 
when  students  began  to  evaluate  courses  and  instructors  in  the  1960s.  Instructors  feared 
backlash  and  still  do,  in  my  opinion.  Mitchell  says,  however,  that  there  is  no  fixed  correlation 
between  grades  and  instructor  evaluation.  This  bears  out,  at  least  in  my  own  situation. 
Mitchell  (1998)  discusses  inflation,  what  grades  mean,  their  value,  and  that  “grades  should  be 
more  than  the  “happenstance  collection  of  idiosyncratic  evaluations”  (p.  128).  Mitchell 
(1998)  establishes  that  grades  should  really  reflect  genuine  discriminations  among  individual 
student  efforts;  all  other  considerations  are  secondary.  He  suggests  that  a rational  exchange 
across  faculty  members  and  instructors  will  fix  the  problem  and  provides  faculty  members 
with  their  historical  grading  averages.  And  even  though  there  are  classes  that  may  actually 


47 


have  all  or  a large  group  of  superlative  students,  more  often  grading  follows  a predictable 
pattern.  He  notes  that  old  habits  die  hard  and  that  we  must  think  seriously  about  grading  and 
improve  the  inflation  of  grades.  Although  this  may  not  be  perceived  as  directly  related  to 
student  demographics,  to  me  it  is  directly,  or  at  least  indirectly,  related.  Students  come  to  me 
as  seniors,  and  if  they  have  received  A’s  to  date  with  widely  varying  course  requirements, 
especially  less  rigorous  ones,  little  feedback,  or  little  critical  feedback  about  the  level  of  their 
work,  and  if  their  courses  have  few  required  tests  or  products,  then  they  have  high  grade 
expectations.  Also  if  the  grades  they  have  received  are  not  clear  in  meaning  and/or  if  the 
grades  they  have  received  are  inflated,  then  my  job  is  very,  very  messy. 

Mitchell  (1998)  notes,  very  important  to  our  initiative,  that  despite  the  high  academic  rigor, 
students  were  still  attaining  good  grades,  even  though  coming  to  college  less  well  prepared 
and  with  a growing  need  for  remediation.  Therefore,  we  included  reading  and  discussion 
about  what  grades  mean  as  a program  component.  We  had  serious  discussions  about 
“curving”  grades  and  what  it  means.  We  ended  by  committing  to,  at  least  for  the  classroom 
experiment,  set  grading  standards  with  an  understanding  that  a professor  not  make  test 
instead  they  ensure  course  and  requirement  rigor  and  teach  differently  to  move  the  students 
toward  achievement  at  higher  levels.  We  have  only  just  begun  but  hopefully  have  already 
made  a difference  with  seven  faculty  members,  some  of  whom  have  established  and 
appropriate  standards.  The  questions  inherent  in  inflated  grading  are  really  about  standards, 
expectations,  and  teaching.  Once  this  becomes  clear,  faculty  members  feel  more  comfortable 
with  higher  expectations  and  established,  yet  transparent,  standards  of  performance.  This 
reduces  their  inclination  to  change  them  “on  the  fly”  to  make  students  appear  more 
successful.  We  do  have  faculty  members  concerned  about  keeping  students  happy,  and  in  all 
fairness  to  these  same  professors,  they  are  concerned  about  not  being  able  to  really 
discriminate  between  several  points  that  may  make  a grade  level  difference  - rightfully  so 
unless  clear  grading  standards  are  in  place. 

This  was  a brief,  but  critical,  program  component.  It  focused  on  helping  the  professors 
understand  how  important  the  development  of  clear  grading  criteria  is  for  student 
assessments,  for  the  professor,  and  for  collecting  evidence  of  learning  that  has  integrity  for 
the  student,  course,  and  program.  There  was  a great  need  with  our  group  to  dispel  the  myths 
about,  and  perceived  benefits  of,  “curving”  student  assessment  grades  or  course  grades.  It 
was  fairly  difficult  to  move  some  of  the  professors  away  from  “curving”  or  widely  subjective 
grading  acts.  However,  there  was  great  progress,  and  they  agreed  to  stick  with  more 
prescribed,  definitive,  and  transparent  grading  strategies  for  their  assessments  and  somewhat 
for  the  course.  This  commitment  was  difficult  because  they  wanted  to  assist  students  in  being 
successful,  but  using  their  methods  made  the  grade  results  for  both  assessments  and  the 
courses  suspicious.  However,  the  newly  developed  courses  with  all  the  direct  connections 
and  mapped  outcomes  to  assessments,  etc.  led  the  professors  to  understand  that  when  the 
connections  are  evident  and  students  can  clearly  see  what  they  are  responsible  for  achieving, 
with  more  opportunities  to  provide  evidence  of  learning  and  clear  levels  of  potential 
performance,  there  really  is  no  need  for  “easy”  assessment  items  or  the  “curving”  of  grades. 
If  the  professors  further  explore  student  self-assessment  (e.g.,  where  students  complete  the 
rubrics  before  turning  them  into  the  professors  for  scoring  or  create  portfolios  to  monitor 
their  own  learning  and  the  more  structured  group/peer  work),  students  will  evolve  even 


48 


further.  Although  briefer,  but  inherently  a part  of  all  the  assessment  sessions,  this  was  a 
critical,  necessary,  and  beneficial  program  component.  All  of  the  professors  changed  their 
assessments  and  grading  strategies  for  the  2006  redeveloped  course  (About  Grades,  n.p.; 
Anderson  & Speck,  1998;  Dominowski,  2001;  Frye,  1994;  Royse,  2001). 

Teaching  Models 

(See  PowerPoint  Presentation  - D7  and  Portfolio  Section  - B 1 1) 

By  the  time  we  focused  on  teaching  models  as  a formal  program  component,  professors  had 
already  become  quite  knowledgeable  about  the  models  below.  The  exposure  resulting  from 
initial  analyses  of  their  2005  course,  described  above,  served  to  more  than  familiarize  them 
with  the  models.  Although  that  analysis  was  driven  somewhat  less  deeply,  it  provided  them 
the  chance  to  learn  about  teaching  models  and  styles  while  comparing  their  current  practices 
against  a list  and  description  of  models  and  styles.  The  worksheet  below  engaged  them  in  a 
more  “studied”  experience,  where  they  learned  more  about  each  model  and  were  asked  to 
consider  how  to  use  each  one  in  their  redeveloped  2006  course.  Initially,  the  strategy  of 
having  them  refer  to  handouts  with  descriptions  of  each  model  and  style  worked  to  inform 
them  well  enough  to  make  judgments  about  their  2005  practices,  but  the  underlying  cognitive 
theory  was  not  revealed  at  that  level  of  examining  the  models.  Therefore,  when  we  reached 
the  stage  of  the  program  where  they  were  to  make  instructional  decisions  about  which 
teaching  models  and  styles  to  incorporate,  more  depth  was  needed,  especially  regarding  the 
cognitive  psychology.  It  was  important  for  them  to  realize  that  if  they  incorporated  a wider 
repertoire  of  teaching  models  and  styles,  students  could  extend  their  primary  learning  style 
comfort  zone  to  include  learning  equally  well  across  a variety  of  learning  styles.  Also  if  a 
more  diverse  selection  of  models  and  styles  was  used,  students  could  more  actively  achieve 
learning  to  higher  cognitive  levels.  Although  a presentation  was  prepared,  because  the 
professors  were  already  so  familiar  with  teaching  models  and  styles,  there  was  no  need  to  use 
it  at  that  point.  Instead  it  was  provided  for  them  to  review.  They  took  the  book  by  Joyce  et  al. 
(2004)  and  the  following  questionnaire  home  and  were  asked  to  engage  in  critical  reflection 
about  teaching  models  beyond  the  initial  level  by  actually  studying  the  models  and  the 
underlying  cognitive  learning  psychology.  In  this  activity,  they  were  to  identify  each  model, 
describe  what  it  could  mean  for  their  teaching  practice,  and  note  how  it  could  change  their 
practice  and  the  2006  course.  Rather  than  do  this  in  class,  we  gave  them  the  day  at  home.  At 
that  point  in  the  program,  we  were  into  summer  intersession,  so  a day  at  home  was  welcome. 
This  exercise  seemed  to  deepen  their  understanding  of  the  models;  thus,  they  broadened  their 
repertoire  of  teaching  models  and  made  choices  about  which  ones  to  incorporate  into  their 
redeveloped  2006  course  instruction.  Joyce  et  al  (2004)  presents  the  families  of  teaching 
models  within  the  context  of  meta-cognition  and  uses  inquiry  and  constructivism  as  the 
process  for  learning.  Scaffolding  is  represented  by  the  stairs  to  show  gradual  and  progressive 
learning  while  constructing  knowledge.  The  learning  context  also  represents  student  learning 
communities  in  which  students  are  engaged  in  zones  of  proximal  development,  an 
environment  where  they  can  stretch  and  grow  without  being  overwhelmed  or  frustrated. 
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Figure  A. 5. 13:  The  Learning  Environment  (See  Portfolio  Section  Bll) 

The  Learning  Environment:  Modeis  of  Teaching 


student  Learning  Communities 


^onceptualX 

Oevelopmei^ 


Student  Teams  of  Expert  Learners 


Scarborough,  2007  (Teaching  Models  by  Joyce  et  al,  2004) 


Figure  A. 5. 14:  Teaching  Model  Map  for  Technology  496  Learning  Environment 

The  Learning  Environment:  Models  of  Teaching 


Tech  496-Scarborough,  2007  (Teaching  Models  by  Joyce  et  al,  2004) 
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The  chart  below  served  as  the  simple  worksheet  for  the  professors  to  further  and  more  deeply 
study  teaching  models  before  making  choices  for  their  courses.  After  the  professors 
completed  their  worksheet,  they  discussed  their  individual  responses  as  a group,  and  where 
necessary,  explanations  to  further  inform  or  illustrate  models  were  made  by  peers  or  the 
program  leader.  Each  professor  then  made  the  instructional  decisions  on  teaching  models  and 
styles  for  his/her  course,  keeping  in  mind  how  they  wanted  to  stimulate  students  to  use  a 
variety  of  learning  styles.  It  might  be  important  to  note  that  we  encouraged  the  professors  to 
select  several  to  try  in  the  2006  course  but  not  to  overwhelm  themselves  with  choosing  too 
many  new  models  to  try  all  at  once,  further  encouraging  them  to  gradually  try  more  models. 
However,  we  also  explained  that  some  of  the  models  are  natural  to  particular  content  or 
teaching  situations  and,  therefore,  not  to  miss  “natural”  opportunities  to  formalize  the  use  of 
any  model  (Fry  et  al.,  2003;  Joyce  et  al.,  2004;  Kolb,  1984). 


Table  A. 5. 14:  Foundation:  Concepts  that  apply  to  Learning  (See  Portfolio  - BIO) 


Concepts  - Chapter  1,  p.3 

Description 

Meaning  for 
Me  and  my 
Practice 

Changes  I will  make 
based  upon  my 
understanding  of  this 
concept? 

Where  will  these  changes 
“show  up”  in  the  teaching  and 
learning  experiences 
throughout  the  semester? 

Constructivism,  p.  12 

Metacognition,  p.  14 

Scaffolding,  p.l4 

Zone  of  Proximal  Development,  p.  16 

Roles  of  Expert  Performance,  p.  20 

Table  A. 5. 15:  Models  of  Teaching 

I.  Models  of  Teaching  - Information  Processing  Models  (See  Portfolio  - BIO) 


Models 

Description 

Strengths 

Weaknesses 

How  I can  use  this 
model-describe 

Inductive  thinking, 

Ch.  3,  p.  41 

(classification-oriented) 

Concept  Attainment,  Ch.4,  p.  59 
(includes  concept  formation) 

The  Picture-Word  Inductive  Model 
Ch.  5,  p.  77 

Scientific  Inquiry, 
Ch.  6,  p.lOl 
Inquiry  Training 

Mnemonics,  Ch.  7, 
p.  131 

(memory  assists) 

Synectics,  Ch.  8,  p.  155 
(includes  metaphoric  activity) 

Advance  Organizers,  Ch.  9,  p.  187 
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Table  A. 5. 15:  Models  of  Teaching  (continued) 

II.  Models  of  Teaching  - Social  Models 


Model 

Deseription 

Strengths 

Weaknesses 

How  I can  use  this 
model-deserihe 

Partners  in  Learning, 
Ch.  10,  p.  205 

Positive  Interdependence,  p.  211 

Struetured  Inqniry,  p.  221 

Group  Investigation  pp.  213-227 

Role  Playing,  Ch.  11,  p.  229 

Jurisprudential  Inquiry,  Ch.  11,  p.  249 

III.  Models  of  Teaching  - Personal  Family 


Model 

Deseription 

Strengths 

Weaknesses 

How  I can  use  this 
model-deserihe 

Nondirective  teaching,  Ch.  12,  p.  271 

Enhancing  Self-esteem,  Ch.  13,  p.  283 

Coneeptual  Development 
Ch.  13,  p.  290 

IV.  Models  of  Teaching  - Behavioral  Models 


Model 

Description 

Strengths 

Weaknesses 

How  I can  use  this 
model-deserihe 

Mastery  Learning,  Ch.  14,  p.  303 
Programmed  Schedule,  p.  310 
Programmed  Schedule,  p.  311 
(task  performanee  reinforcement) 

Direct  Instruction,  Ch.  15,  p.  313 

Simulation,  Ch.  16,  p.  323 
Training  and  Self-Training 

Soeial  Learning,  Ch.  14 
(includes  training  & self-training) 

Although  the  professors  clearly  found  they  relied  heavily  on  lectures  in  the  2005  courses, 
some  of  them  did  use  other  models  but  not  to  the  degree  of  variety  that  was  desired  once  they 
analyzed  their  courses.  After  they  more  deeply  “studied”  the  models  and  made  choices  about 
the  ones  they  wanted  to  incorporate  into  their  delivery  strategies  for  the  course,  they  also 
developed  the  course  schedule.  When  completed,  the  schedule  was  modified  to  provide 
columns  where  they  could  note  the  teaching  models  and  styles  they  felt  would  work  during 
the  weeks  and  days  of  the  course.  This  was,  once  again,  an  exercise  that  provided  an 
opportunity  to  more  critically  consider  when  they  could  actually  “plan”  to  use  various 
models  and  styles,  especially  the  ones  they  had  not  used  before.  It  also  provided  the 
integrated  activity  to  see  the  relationship  between  chosen  teaching  models  and  teaching 
styles.  Furthermore,  it  illuminated  the  connection  between  teaching  models  and  styles  and 
student  learning  styles  and  how  that  could  influence  their  potential  achievement  of  Bloom’s 
higher  cognitive  dimensions  and  Dale’s  more  active  learning  levels.  Although  somewhat  an 
“exercise,”  this  activity  brought  about  the  connection  between  “planning  and  consideration 
of’  and  “actual  use  of’  these  models  and  styles  to  achieve  higher  student  engagement  and 
cognitive  processing.  It  clearly  required  formal  decisions,  and  the  visual  was  a good  method 
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to  establish  commitment  to  those  decisions.  Creating  the  course  calendar  led  us  into  the 
syllabus  program  component,  as  a redesigned  2006  syllabus  was  a requirement  for  each 
professor.  It  might  be  important  to  note  that  some  professors  left  the  calendar  showing  the 
models,  etc.  rather  than  using  that  version  for  themselves  and  using  the  calendar  without  the 
information  on  the  syllabus  for  the  students.  This  made  a somewhat  messy  syllabus  and  was 
not  really  recommended  by  the  program  directors.  However,  at  that  point  in  the  program  (the 
end),  they  were  exhausted  but  pleased  with  their  products  and  new  instructional  decisions. 
The  legend  for  the  form  is  below  as  well.  This  learning  activity  led  to  the  proper  component 
development  of  the  2006  syllabi. 


Table  A.5.] 

6:  Course  Calendar:  Topics,  Class  Schec 

ule.  Due  Dates  (2  day /week  course) 

Week  & 
Outcomes 

TM 

Dale 

TS 

LS 

Day  1 

Topics,  Activities,  Due  Dates 

TM 

Dale 

TS 

LS 

Day  2 

Topics,  Activities,  Due  Dates 

Week 

Dates 

Week  2 
Dates 

Table  A. 5. 17:  Course  Calendar  for  Teaching/Learning  Models  and  Styles  Legend 


Teaching  & Learning  Calendar  Legend 

Teaching  Models 

Teaching  Styles 

Kolh’s  Learning  Styles 

(Joyce,  Weil,  Calhoun,  2004) 

(Mosston  & Ashworth,  1990) 

(Kolh,  1984) 

IT  - inductive  thinking 

C - command  (A) 

CE-concrete  experience 

CA  - concept  attainment 

P - practice  (B) 

RO  - reflective 
observation 

PWIM  - picture  word  induction  model 

R - reciprocal  (C) 

AC  - abstract 
conceptualization 

Scl  - scientific  inquiry 

sc  - self  check  (D) 

AE  - active 
experimentation 

M - mnemonics 

I - inclusion  (E) 

S - synectics 

GD  - guided  discovery  (F) 

Eelder’s  Learning  Stvles 

AO  - advance  organizers 

CD  - convergent  discovery  (G) 

active  vs.  reflective; 

Partners 

DP  - divergent  production  (H) 

sensing  vs.  intuitive 

CL-I  - cooperative  learning-informal 

ED  - learner  designed  (I) 

visual  vs.  verbal 

CL-F  - cooperative  learning-formal 

LI  - learner  initiated ) J) 

sequential  vs.  global 

SI  - structured  inquiry 

ST -self  teach  (K) 

GI  - group  investigation 

Bloom  (1956) 

RP  - role  playing 

Teaching  Stvles 

K-R  - knowledge  or 
remember 

JI  - jurisprudential  inquiry 

(Grasha,  1996) 

C-U  - comprehension- 
understanding 

NT-nondirective  teaching 

E - expert 

Ap  - application  or  apply 

ES  - enhancing  self-esteem 

EA  - formal  authority 

An  - analyze 

ML  - mastery  learning 

PM  - personal  model 

S-E-  synthesize  or 
evaluate 

PS  - programmed  schedule 

F - facilitator 

E-C  - evaluate  or  create 

DI  - direct  instruction 

D - delegator 

Dale’s  Cone  of  Learning 
(1969) 

S - simulation 

P=listening; 

I-participation; 

A=doing 
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Syllabus 


Professors  were  asked  to  analyze  their  syllabi.  As  the  most  critical  communication 
instrument,  we  addressed  redevelopment  of  the  syllabus  equally  beside  all  the  teaching, 
learning,  and  assessment  program  components.  Since  the  syllabus  is  supposed  to  inform 
students  about  the  course,  professor  expectations,  and  their  responsibilities,  it  clearly  needs  to 
communicate  effectively.  Such  a syllabus  should  provide  enough  information  so  if  students 
have  choices  about  which  semester  is  best  for  them  to  enroll  in  a particular  course,  they  can 
make  better  choices  because  the  information  is  available.  CEET  requires  our  course  syllabi  to 
be  on  file  and  available  to  students,  and  although  good  courses  are  usually  modified,  at  least 
somewhat,  each  time  they  are  offered,  based  upon  student  feedback  or  new  content,  etc.,  the 
recent  past  syllabus  can  inform  a student  well  enough  for  more  informed  decision  making. 

There  are  many  different  styles  of  presenting  information  on  a syllabus  and  just  as  many 
different  preferences  across  professors  about  what  to  include  on  a syllabus.  The  philosophy 
used  for  the  program  was  that  students  are  entitled  to  full  disclosure  about  the  course  and  that 
the  course  should  be  planned  in  great  detail  where  few  changes,  if  any,  are  to  take  place. 

Only  something  unusual  should  result  in  a change,  and  changes  would  usually  be  to  the 
students’  benefit,  so  there  should  be  no  surprises  or  major  changes  in  the  course,  its  work, 
assignments,  due  dates,  etc.  This  means,  very  simply,  that  professors  must  plan  the  course  in 
its  entirety.  Although  we  had  an  excellent  example,  the  program  leader  reviewed  the 
literature  to  see  if  there  were  any  suggestions  that  might  further  enhance  the  one  we  were 
using  as  a model.  However,  the  literature  review  confirmed  that  ours  was  an  excellent 
example,  so  we  provided  some  of  those  articles  to  the  professors  for  validation  purposes. 
Using  that  exemplar  and  those  articles,  we  operated  with  a bias  that  students  should  expect  to 
receive  full  course  disclosure  on  the  course  through  a syllabus  that  reveals  the  following: 
(Campbell  & Smith,  1997;  Dominowski,  2002;  Eorsyth,  2004;  Killian,  1995;  Royse,  2001; 
Scarborough,  2006  - Tech  496). 

• Professor/Graduate  Assistant  Contact  Information:  All  information  needed  for 
appropriate  student-professor  contact,  clearly  making  it  possible  for  the  students 
to  be  able  to  predictably  get  in  touch  with  the  professor  and  graduate  assistant 
outside  of  class. 

• Catalog  Course  Description:  The  course  description  taken  from  the  college, 
department  website,  or  catalog.  The  description  should  make  clear  all  pre- 
requisites and  focus  or  overall  course  objectives.  If  the  course  objective(s)  or 
outcomes  are  not  clearly  inherent,  it  might  be  important  to  actually  state  the 
course  objectives  separately  to  link  to  the  student  learning  outcomes  mentioned 
below. 

• Course  Purpose:  Although  some  feel  that  this  is  unnecessary,  when  used 
appropriately,  it  explains  to  students  why  the  course  is  required  or  important,  what 
it  will  do  for  them,  and  its  context.  When  the  purpose  is  described  appropriately, 
students  understand  what  knowledge  and  skills  they  will  learn  from  the  course, 
what  the  knowledge  or  skills  will  do  to  contribute  to  their  overall  plan  and 
program,  and  finally  - possibly  most  importantly,  it  creates  the  context  for 
learning  what  the  course  offers,  thus  leading  the  professors  to  its  importance  in 
gaining  employment  professionally. 
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• Requirements:  This  makes  very  clear  what  students  need  or  can  expect  that  may 
or  may  not  be  normally  assumed  by  students  (e.g.,  text,  calculator,  computer, 
transportation,  special  labs,  mention  of  special  projects,  etc.).  For  example,  in 
Tech  496,  a lab  course  that  meets  six  hours  each  week,  students  are  also  required 
to  meet  in  teams  an  additional  number  of  hours  each  week.  Although  it  is  best  to 
publish  this  type  of  requirement  along  with  times  and  days  on  the  web  for 
registration,  it  clearly  needs  to  be  addressed  the  first  day  of  class  and  published  in 
the  syllabus  as  the  requirement  affects  student  schedules  overall. 

• Expected  Computer  or  Technology  Skills:  Knowledge  or  experience  with 
particular  software,  mechanical  skills,  design  skills,  computer  skills,  etc.  For 
example,  most  of  our  students  are  assumed  to  know  how  to  search  and  use  the 
Internet  as  a research  tool;  however,  we  have  often  found  that  students  have  good 
skills  in  particular  uses  or  searches,  but  that  we  cannot  assume  those  skills  cross 
over  a wider  variety  of  searches  and  uses.  They  assume  that  Google,  etc.  will 
provide  them  access  to  everything  they  need,  when  in  fact,  particular  search 
engines  or  indexes  lead  to  different,  “best,”  resources,  etc. 

• Pre-requisites:  Other  courses  required  before  taking  this  one  or  information  such 
as  “senior  status.” 

• Course  Requirements  (assessments,  assignments,  etc.)  and  Grading  Information: 
This  is  a list  of  each  assignment,  assessment,  and  activity  that  contributes  to  the 
course  grade.  Grading  is  presented  so  students  understand  the  percentages  or 
points  per  assignment  and  the  weight  each  holds  as  part  of  the  total  final  course 
grade. 

• Student  Learning  Outcomes:  Clear  listing  of  ABET/TAC/NAIT  standards  or 
outcomes  and  their  connections  to  NIU’s  General  Education  Goals  as  well  as  the 
connections  to  student  assessments.  This  presents  clearly  in  outcome-oriented 
statements  what  they  are  going  to  learn  or  be  able  to  do  and  how  they  will  be 
assessed  for  evidence  of  learning,  while  clearly  revealing  the  “assumed” 
knowledge  and  skills  resulting  from  the  embedded  General  Education  knowledge 
and  skills. 

• Course  Schedule:  A very  clear  presentation  of  each  course’s  weeks  and  days, 
revealing  the  topics  to  be  covered,  student  activities,  any  speakers,  DVDs, 
fieldtrips,  and  all  assignments  and/or  assessments  and  their  due  dates.  Identify 
special  labs  or  meetings,  holidays,  etc.  This  should  reflect  the  entire  schedule  and 
reveal  what  is  going  to  occur  or  be  due  on  each  class  day  - the  Plan  and  the 
Contract  between  student  and  professor. 

• Course  Requirements  Explanation:  This  describes  each  item  listed  in  the  Course 
Requirements  above  and  specifies  what  is  expected  of  the  student.  It  identifies 
and  describes  any  particular  strategies,  processes,  sources,  formats,  required 
software;  whether  it  is  a team  or  individual  process  or  product;  and,  refers  to 
rubrics,  handouts,  etc.  Each  description  should  make  the  assignment,  activity, 
requirements,  or  assessments  clear  to  the  student  and  should  be  broken  out  into 
relational  categories  so  that  students  can  see  how  one  type  of  learning  activity 
relates  to  another.  It  helps  to  use  a descriptor  beside  each  category  title.  Visually, 
it  helps  if  these  are  boxed.  (See  example  below.)  Finally,  team  versus  individual 
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course  requirements  should  be  identified  separately  and,  when  possible,  reveal 
how  one  leads  to  the  other,  if  appropriate. 

• NIU  Academic  Misconduct  Policy:  NIU’s  policies  on  academic  misconduct  and 
the  process  the  professor  will  implement  in  class  regarding  any  academic 
misconduct. 

• Professor’s  Notes:  Any  specifics  about  course  requirements,  behavior, 
tardy/absence  policies  or  policies  regarding  dress,  cell  phones,  use  of  other 
technology,  behavior,  eating/drinking,  and  others  specific  to  each  professor. 

• Student  Support  Services:  Where  students  can  seek  academic  support  (e.g.,  NIU’s 
accessibility  services,  tutoring.  Writing  Center,  etc.),  whether  provided  by  specific 
NIU  offices,  the  department,  or  the  college. 

• Course  References  (library  reserve):  A listing,  location,  and  availability  specifics 
of  any  resources  being  held  on  reserve  for  students  in  a particular  course  or  on  the 
Website  and  how  to  access  the  Website. 

• Course  Requirements/Points  Check  Off/  and  Notes:  It  is  important  for  students  to 
monitor  their  own  learning  progress.  To  help  that  along,  include  a Check  Off 
Form  that  lists  each  assignment,  activity,  or  assessment,  its  point  or  percentage 
value,  and  due  dates  if  preferred  - separating  individual  versus  team  items. 
Require  students  to  monitor  themselves  and  check  their  progress  by  filling  in  the 
grades  or  scores  they  have  acquired  as  each  item  is  completed,  then  their  progress 
to  their  goal  grade.  This  type  of  form  is  used  for  that  portfolio  assessment  as  well. 

• Grading  Parameters:  Grading  values  are  clearly  specified  (e.g.,  how  many  points 
or  what  percentage  results  in  an  A,  B,. . .);  however,  I have  also  included  an 
atypical  parameter.  Using  the  term  “Benchmark”  to  mean  a student  or  product, 
behavior,  or  process  that  is  identified  as  one  others  would  want  to  achieve,  an 
example  of  the  best,  is  represented  in  the  grading  as  99-100%,  a “benchmark  for 
others.”  This  seems  to  motivate  students,  whether  younger  or  older, 
undergraduate  or  graduate,  toward  higher  achievement.  No  longer  are  A’s  the 
goal  in  my  classes.  The  students  work  harder  to  achieve  the  Benchmark  status, 
although  it  is  really  no  different  than  what  has  traditionally  been  available  as  a 
higher  A status,  but  somehow,  terming  the  highest  level  as  “Benchmark”  is  very 
motivating. 

Syllabus  Model 

Below  is  the  example  professors  used  for  comparison.  The  syllabus  is  for  the  senior  design 
capstone  course  in  technology.  The  course  integrates  senior  design  with  project  management 
and  leadership  through  formal  teams.  It  is  the  culminating  capstone  course  and  results  in 
formal  individual  student  and  team  portfolios  with  complete  evidence  of  learning  for 
employers. 
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Figure  A.5.12:  Technology  496  - Industrial  Project  Management 

Technology  496  - Industrial  Project  Management 

Prof: Grad.  Asst: Ph:  753-0210  Off.Hrs:  T12-3  Email: 

I.  Catalog  Course  Description;  Industrial  Project  Management  (3).  Basic  concepts,  principles,  and  skills  of  project 
management.  Designed  to  cover  a variety  of  types  of  project  management.  Emphasis  on  computer  tools  and  project  management 
techniques.  Analysis  of  case  studies.  Culminating  project  required. 

II.  Course  Purpose  & Objectives:  To  prepare  project  leaders  and  team  members  to  formally  initiate,  execute  and  terminate 
industrial  projects  effectively.  To  integrate  and  apply  knowledge,  skills,  and  abilities  acquired  or  extended  during  students’  college 
careers  (general  education  and  major)  and  work  experience  to  research,  design,  build  and  finalize  a technical  project  within  a team 
and  formal  project  environment. 

III.  Required  Text:  Project  Management.  Cleland  & Ireland,  2006  or  latest  edition. 

Required;  Date  book/Calendar  for  scheduling  and  notes;  Handout  packet. 

IV.  Pre-requisites:  Tech  265-Mfg.  Processes;  Tech  302-  Graphic  Pres.&  Comm.;Tech  395-Ind.  Data  Processing;  Senior  Status 

Expected  Computer  Usage:  CAD,  MS  Office,  MS  Project,  CNC,  industrial  equipment,  or  other,  depending  upon  semester/  team 
project.  Required  Laboratory  Team  Project;  Changes  each  semester;  each  team  will  engage  in  a complex  technical  project 
with  specific  technical  standards  to  achieve,  e.g..  Go-kart,  3-car  passenger  train,  hovercraft,  paddle  wheel  boat,  personal  transport 
vehicle  etc.  Research,  design,  assembly  of  electrical/mechanical  systems,  testing,  modifications/finalization  with  formal 
documentation,  formal  team  products  and  team  requirements.  See  requirements  section,  handouts,  and  rubrics. 


V.  Course  Requirements 


Individual  Course  Requirements: 

Points: 

Team  Course  Requirements:  Points: 

Grading: 

Text  Project  (broken  into  sections/due  dates) 

7 

Team  Operations  Manual 

5 

Benchmark 

Project  Research 

5 

Community  Leadership  Service 

3 

A=98-100 

Project  Design 

5 

Project  & Articles 

Literature/Internet  Research  Tables  A & B 

7 

Team  Project  Plan 

5 

A=93-97 

(Proj  s/T  ms/Lead/Int  ’ 1/MCT  ms/MCLd) 

Team  Project  & Assessment 

7 

B=92-85 

Career  Project 

5 

(Final  Exam) 

C=  84-77 

Individual  Case  Study 

5 

*Peer  Assessment  Process 

5 

D=76-70 

Paper 

7 

& Team  Success  Assessment 

p=69-below 

Midterm:  Individual  Project  Plan 

7 

*Team  Member  Participation 

Software  Workshop/Test 

5 

(Ind.  Pts.  3/5) 

*Team  Participation  Awarded  by  Team 

3 

Team  Presentation  & Success 

5 

Project  Feedback  Logs 

1 

(Pinal  Exam) 

Individual  Portfolio  & Assessment  Process 

3 

Team  Project  Portfolio  & Website 

5 

(Final  Exam) 

Professor’s  Overall  Assessment 

Pinal  Exam  II:  TBD 

Ind.  Presentation  within  Team  Presentation. 

5 

(if  needed  to  confirm  competencies) 

Total  Individual  Points  Possible 

65 

Total  Team  Points  Possible 

35 

Professor’s  Privilege 

See  Note  #2 
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VI.  Student  Learning  Outcomes 


Student  Learning  Outcomes 


Embedded  NIU  General  Embedded  NAIT/ABET 
Ed  Goals  Learning  Standards 


lA/B.  Identify  and  describe  major  problems, 
issues,  concerns,  and  solutions  (PICS)  that 
relate  to  (a)  projects,  (b)  project  management, 
(c)  project  teams,  and  (d)  project  leaders,  also 
for  (e)  Int’l  projects  and  (f)  multicultural  (MC) 
teams. 

2.  Identify  and  describe  best  practices  for 
managing  projects  and  leading  teams;  include 
Int’l  teams  and  MC  teams. 


3a.  Demonstrate  effective  project  management 
of  a technical  project  using  appropriate  PM 
techniques,  tools,  and  processes: 

a.  planning, 

b.  initiation, 

c.  execution, 

d.  termination 

e.  evaluation 

f.  problem  solving 

g.  leadership 

h.  Hnancial  management 

i.  procurement  management 

j.  scheduling 

k.  MACE  process  and  procedures 

3b.  Design,  develop,  and  deliver: 

e.  executive  team  presentation 

f.  team  portfolio 

g.  team  website 

4.  To  integrate  mathematics,  the  sciences, 
English,  management,  technical,  technological 
systems  knowledge  and  skills  to  accomplish 
individual  and  team  project  objectives: 

(a)  Design,  (b)  Build  a vehicle  to  technical 
specification  that  will  operate;  (c)  Solve 
technical  problems  encountered;  (d)  test  and 
evaluate  the  vehicle  for  meeting  technical 
specifications  and  standards 


a.i.  communicate  clearly  in 
English,  demonstrating  ability  to 
comprehend,  analyze  and 
interrogate  critically; 
ii.  communicate  in  a manner 
that  unites  theory,  criticism, 
practice  in  speaking  & listening; 
a.iv.  Aware  of  and  able  to  use 
various  resources,  including 
modem  technology 


a.  cultivate  habits  of  writing, 
speaking,  quantitative  reasoning 
for  continued  learning: 

a.i.  communicate  clearly  in 
English,  demonstrating  ability  to 
comprehend,  analyze  and 
interrogate  critically; 
ii.  communicate  in  a manner  that 
unites  theory,  criticism,  practice 
in  speaking  & listening; 
a.iii.perform  basic  computations, 
display  facility  with  use  of 
quantitative  reasoning  in  forming 
concepts  for  analysis  and  in 
problem  solving,  and  interpret 
mathematical  models  and 
statistical  info 

a. iv.  Aware  of  and  able  to  use 
various  resources,  including 
modem  technology 

b.  develop  an  ability  to  use 
modes  of  inquiry  across  a variety 
of  disciplines  in  the  physical 
sciences,  mathematics: 

b. iii.  demonstrate  ability  to  use 
scientific  methods,  theories  to 
science  phenomena; 

c.  develops  understanding  of 
discipline  interrelatedness, 
applying  that  knowledge  to  an 
understanding  of  important 
problems  & issues. 

d.  develops  social  responsibility 
& preparation  for  citizenship 
through  service  and  an 
appreciation  of  cultural  diversity 


g.  demonstrate  an  ability  to 
communicate  effectively  in 
writing; 

h.  demonstrate  an  ability  to 
communicate  effectively 
orally; 

m.  demonstrate  an  ability  to 
. . . .utilize  computer 
applications  effectively; 

k.  demonstrates  a respect  for 
diversity  and  knowledge  of 
contemporary  professional, 
societal  and  global  issues. 


a.  demonstrate  appropriate 
mastery  of  knowledge, 
techniques,  skills,  and  modem 
tools  of  the  discipline; 

b.  demonstrate  ability  to 
apply  current  knowledge  and 
adept  to  emerging 
applications  of  math,  science, 
engineering  and  technology; 

d.  demonstrate  ability  to 
apply  creativity  in  the  design 
of  systems,  components  or 
processes  appropriate  to 
program  objectives; 

f.  demonstrate  ability  to 
identify,  analyze,  and  solve 
technical  problems; 
g-h.  demonstrate  ability  to 
communicate  effectively  in 
writing  and  orally; 

l.  demonstrate  commitment 
to  quality,  timeliness,  and 
continuous  improvement; 

m.  demonstrate  ability  to 
. . . .utilize  computer 
applications  effectively; 

o.  demonstrate  an  ability  to 
manage  projects,  industrial 
systems,  lead  personnel 
effect. 

p.  demonstrate  an  ability  to 
manage  and  manipulate 
industrial  systems; 

q.  demonstrate  knowledge, 
strategies  and/or  techniques  of 
how  to  lead  personnel  and 
teams  effectively 

e.  demonstrate  ability  to 
function  effectively  on  teams; 

j.  demonstrate  ability  to 
understand  profess-ional, 
ethical,  social  responsibilities; 

k.  demonstrate  respect  for 
diversity,  knowledge  of 
contemporary  professional, 
societal  and  global  issues 


Assessments/Rubrics 

Text  Project  or  Text  Test 
Research-  Literature/Intemet; 

Case  study; 

Group  analysis  process 
Formal  paper;  group  analysis 
1-5  minute  learning  papers; 

Individual  portfolios; 

Team  Project  portfolio/website; 
Individual/team  presentations 
Team  participation  & Peer  Assessment 
Team  Operating  Manual 
Individual  and  Team  Project  Plan 
Community  leadership  project 

Individual  & Team  project  research 
Individual  & Team  project  design 

Written  individual  & team  plan(s); 

Technical  project  prototype  product 
produced  to  technical  standards  and 
specifications  using  technical  processes 

Project  testing  & evaluation  against 
established  standards  and  specifications 
using  formal  evaluation  tools  and 
procedures 

MS  Project  2003  test  and  application  in 
project  planning,  execution,  termination, 
assessment  and  evaluation 
MACE-Project  assessment 
(Plan  compliance  & adjustments) 
Individual  & Team  Logs 

Individual  and  Team  Portfolio(s); 
website(s); 

Individual  and  team  presentations; 
Industrial  panel  evaluation 

Project  termination  with  lessons  learned 
Project  evaluation  by  industrial  panel 

Team  Operations  Manual; 

Team  Plan 

Team  presentation;  portfolios;  website; 
Team  peer,,  team,  & conflict 
assessments/logs; 

Industrial  panel  evaluation; 

Formal  paper  ;5  minute  learning  papers 
Team  success  rubric 
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VII.  Topics,  Class  Schedule  & Due  Dates 


Wk/Dte 

Topics 

Date 

Topics/Lab  Act. 

Assignment  Due  Dates 

1 

Course 

Intro 

9:30  Writing  Center  Requirements 
(Jacky) 

9:  45Career  Project  Intro  and  Requirements 
(Norwood) 

10:00  Team  Selection/Scheduling 

10:30  Course  Intro 
1 1 :30  Legacy  Group 

Use  of  Planner  & The  Nature  of  Multitasking 
Project  Research  Assignment 

Teaming 

Team 

Assess. 

Team  Skills  Bank 
Finalize  Teams  & Schedules 

Plan  Team  Service  Project 

Project  Research  Review 

Schedule  Writing  Center 
NOW!!!! 

Due  1/20 
Writing  Center 
Appointments 
Projeet  Researeh 
Bring  Planner 

Community  Serviee  Art.  & 
Plan  Duel/20  4:00pm 

2 

Text  1-4, 
19 

TEXT  Highlights 

Teaming 
*210  sched. 

Project  Design  Lab 

Writing  Center  Appts.  Due 
1/25 

Text  ProJ.  1-4, 19,  20  due 
1/25 

Industry  Case  ID  due  1/25 
Final  Project  Research  due 
1/27 

3 

Teaming 
Text  18 
&HB 

Project  Teams:  hidden  agendas,  teamwork, 
effective  teams  & members,  member  roles  & 
responsibilities  - Peer  Assessment 

(Team  Packet  Required) 

Research 
Design 
*210  sched. 

Project  Design  Lab 

Text  Projeet  18,  21  due  2/1 
Projeet  Design  due  2/4 
Friday 

4 

Teaming 
Text  20 

Project  Teams:  conflict  resolution,  decision- 
making, teams  in  trouble,  empowerment,  trust, 
recognition 

(Team  Packet  Required) 

Teaming 

Team  Manual  Lab 

Lit.  Research  Table  due  2/8 
Career  Project  due  2/11 

5 

Project 

Plan 

Text 

11,6 

Project  Planning  - Section  I Rubric  & TEXT 
Vision,  Mission,  Intro,  Purpose,  Scope, 
Objectives,  Deliverables,  Charter,  Org.  Charts, 
Stakeholder  Analysis,  Com  Interface,  Project 
Review,  Change  Plan  [Paper  due] 

Teaming 

Research,  Case,  Paper,  Career  Validation 
Activity  - Group  Process 

Team  Manual  Lab 

Industry  Case  due  2/15 

Text  Proj.  6,  8, 11, 16  due 
2/15 

Team  Manual  due  2/14 

6 

Project 
Plan 
Text  13, 

Project  Planning  - Section  II  Rubric  & TEXT 
Business  & Proj.  Success  Eactors,  SWOT 
Analysis,  Project  Constraints,  Risk  Analysis, 
Contingency  Plans  & Trade  Offs,  Statement  of 
Work,  Goals,  Work  Break-down  Structure 

Projeet 

Planning 

*210  sched. 

Project  Planning  - Section  II  Lab 

Community  Leadership 
Projeet  and  Articles  due 
2/25  Friday 

Paper  due  2/21 

7 

Project 

Plan 

Section  II  Rubric  & TEXT  (Continued) 

Life  Cycle,  Productivity  Plan, 

Quality  Standards  & Metrics, 

Project  Monitoring,  Assessment,  Control  and 
Evaluation,  Linear  Charts,  Resource 
Plan/Budget  MS  Project  - PM  Software 

Software 

Workshop 

*210  sched. 

MS  Project-PM  Software 

Text  Proj.  5,9, 12, 13, 14, 15 
due  3/1 

8 

Project 

Plan 

Section  III  Rubric  & TEXT 
Environmental/Safety  Plan,  Security  Plan, 
Documentation/Configuration  Mgmt.  Plan, 
Project  Divestment  & Termination  Plan 

Software 
Workshop 
*210  sched. 

MS  Project-PM  software 

Individual  Plans  due  3/11 
Software  Test  due  by  3/9 

BREAK 

3/17 

BREAK 

9 

Project  Development  & Teamwork 

3/24 

Project  Development  & Teamwork  [Logs] 

Individual  Portfolios  due 
3/22 

Team  Plans  due  3/25 

10 

Project  Development  & Teamwork 

3/31 

Project  Development  & Teamwork  [Logs] 

Individual  Portfolios  due 

11 

Project  Development  & Teamwork 

4/7 

Project  Development  & Teamwork[Logs] 

12 

Project  Development  & Teamwork 

4/14 

Project  Development  & Teamwork[Logs] 

13 

Project  Development  & Teamwork 

4/21 

Project  Development  & Teamwork  [Logs] 

14 

Project  Development  & Teamwork 

4/28 

Project  Testing  and  Initial  Assessment 

Proj.  Test./Assess  due  4/28 

15 

[Final  Project  Assessment  & Grade] 

[Peer  Assessments  Executed  & Due] 
[Team  Member  Participation  Determined] 

5/5 

[Team  Presentations  8:30am-12:30] 
[Team  Portfolio/Website/Success  due] 

Team  ProJectAssessment 
due  5/3 

Team  Pres./Port./Web.  5/5 

16 

May 

5/12 

Final  Exam:  TBD 

If  needed  to  confirm  competencies 

Finals 

Week 
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VIII.  Course  Requirements  Explanation  - Individual  Requirements: 

Text  Project:  Read  the  entire  text  and  answer  the  take-home  questions.  You  will  engage  in  a group  process  and  then  participate  in  a non- 
traditional  test  on  this  content  to  ensure  concept  attainment.  Individual  and  Group  Process. 


Technical  Research  and  Design 

Project  Research:  Research  project  assigned.  More  information  about  this  research  will  be  provided  in  class.  However,  it  will  entail  an 
Intemet/Literature  search,  possibly  interviewing  technical  experts,  local  or  suburban  vendors  or  manufacturers,  or  other  professors,  and/or 
researching  specific  technicalities.  It  will  also  include  research  of  all  properties  of  materials,  mathematics,  and  scientific  principles,  theories 
involved  in  the  technical  aspects  of  the  project.  Use  research  information  to  design  the  project.  See  Rubric.  Individual  and  Group  Process. 

Project  Design:  Students  will  design  and  prepare  visuals  and  working  drawings,  schematics,  etc.  for  the  project  using  prior  design  and 
computer  aided  drafting  or  mechanical  drawing  knowledge  and  skills.  See  Rubric.  Individual  and  group  process. 


Real  World  Validation  - Culminating  Paper 

Literature/Internet  Research  A:  Search  the  literature  (Internet)  on  project  management,  project  teams,  and  project  leadership;  identify  45 
quality  sources,  15  each  about  (a)industrial  projects, (b)  project  teams,  and  (c)project  leadership.  Develop  a literature/source  review  Table 
summarizing  what  the  literature/sources  revealed.  Topics  of  focus  should  be  the(l.)problems,  issues,  concerns,  (PICs)  difficulties  that  arise  on 
projects  or  for  the  teams  and  leaders  and  (2.)success  strategies  that  have  worked  for  projects,  project  teams  or  leaders  in  resolving  the 
problems/issues.  There  must  be  45  sources;  these  must  be  from  major  recognized  journals  or  books  on  the  topics.  You  may,  however,  include 
up  to  five  non-traditional  sources,  e.g.,  Internet  sources  from  industrial  groups,  project  teams,  etc.  Sources  must  show  depth  in  content;  short 
“briefs”  are  not  acceptable.  Copy  all  sources  if  not  books  on  diskette  or  CD  rather  than  hardcopies.  See  Table  Format  and  Rubric.  Group 
Process-Be  prepared  to  discuss;  thus,  if  no  hardcopies  available  for  reference,  you  need  to  know  the  material  well.  **Create  tables  that 
are  categorized,  numbered  and  reveal  in-deptb  information  with  solutions.  May  use  for  your  paper. 

Literature  A -i-  B = Total  Table  (See  Rubric) 

Literature/Internet  Research  B:  Also,  research  (a)intemational  projects,  (b)multicultural  teams,  and  (c)intemational  project  leadership 
with  a multicultural  team;  identify  15  (5  for  each  topic)  Internet  and/or  literature  sources  that  discuss  (l.)problems,  issues  and  (2.)  best 
practices,  benefits,  successes  of  multicultural/international  projects,  teams,  and  project  or  team  leadership.  Summarize  the  information 
learned  by  organizing  it  into  a Table  identifying  the  source  author,  title,  main  points  on  problems,  issues,  and  benefits  and  your  comments. 

See  Rubric.  Individual/Group  Process-Be  prepared  to  discuss;  thus,  your  if  no  hardcopies  available  for  your  reference,  you  need  to  know 
the  material  well.  **Create  tables  that  are  categorized,  numbered  and  reveal  in-deptb  information  with  solutions.  May  use  for 
paper. 

Industrial  Case  Study:  Identify  a company  that  will  allow  you  to  visit  and  interview  an  industrial  project  team.  Interview  a project  leader 
or  manager  and  at  least  three  project  team  members  or  2 project  leaders  and  2 project  team  members.  (l.)Ask  them  to  identify  all 
problems,  issues,  concerns,  (PICs)  or  difficulties  encountered  on  the  project,  about  the  project,  team.,  and  project  leadership.  Have  them 
explain  in  detail;  (2.)then,  also  ask  them  what  strategies  are  successful  for  projects,  teams,  and  project  leaders.  Create  a table  of  questions  and 
responses  and  present  what  was  learned  as  “real-time”  research.  See  Rubric  and  Format.  Individual  and  Group  Process.  Incorporate  the 
results  into  your  paper. 

Formal  Paper:  Meet  with  WC  tutor  to  orgauize  paper.  Develop  a paper  about  projects,  teams  and  project  leadership;  develop  the 
issues  and  solutions  in  greater  depth;  draw  conclusions  and  describe  effective  project  management,  effective  project  teams,  and  effective 
project  leadership.  What  strategies,  techniques,  processes  should  be  used  to  have  a more  successful  project,  team,  or  leader/leadership  process? 
End  with  very  specific  recommendations  to  guide  your  project  team  on  each  of  the  3 primary  topics.  Then  include  a section  on  how 
international  projects  and  multi-cultural  teams  differ,  what  additional  concerns,  problems,  and  issues  occur  when  operating  internationally  with 
diverse  cultures.  Make  recommendations  for  successful  international  projects  and  on  how  to  be  a more  effective  leader  of  multicultural  teams. 
Sixty  (60)  sources  required  (45  + 15).  These  60  sources  may  or  may  not  be  the  same  ones  that  you  identified  for  the  literature  review  table. 
**Incorporate  the  results  of  industrial  case  study  into  your  paper  as  well.  Use  the  APA  writing  style  manual.  Identify  all  sources  in 
the  paper’s  text  and  in  References  Cited  using  the  APA  style  format.  Writing  skills  are  seriously  graded  on  this  product.  See  Writing 
Rubric,  Paper  Outline  & Rubric.  Individual  and  Group  Process. 
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Final  Exam 

Team  Project:  Each  team  will  be  responsible  for  designing  and  developing  a technical  project.  You  will  generate  technical  standards  to 
achieve  and  the  metrics  to  use  to  measure  the  standards  achievement  level.  The  project  must  “function”  or  “work”  to  be  accepted  for  a grade.  It 
must  meet  the  standards  at  the  level  described  in  the  team  plan  using  the  metrics  predetermined.  Every  team  member  must  have  major  project 
role  and  responsibilities.  The  team  must  complete  the  project  by  the  deadline  on  the  syllabus.  The  project  is  the  “vehicle”  providing  evidence  of 
high  performance  teaming  and  project  management  as  well  as  the  knowledge,  skills,  and  abilities  from  academic  career  and  work  experience. 
Team  derived/Professor  approved-  predetermined  -Standards/metrics  = grading  Rubric.  Group  Process. 

Team  Portfolio:  The  portfolio  is  the  culminating  documentation  of  all  project  and  team  work.  It  must  include  information  on  every  topic  listed 
in  the  outline/rubric.  It  should  include  pictures,  mechanical  drawings,  etc.  and  be  professionally  produced  in  hard-copy  form.  An  operator’s  and 
maintenance  manual  must  be  developed  and  included  for  the  technical  project  (product).  See  Rubric.  Group  Process. 

Team  Website:  Each  team  is  to  design  and  produce  a team  web-site  which  will  serve  as  an  electronic  portfolio.  This  website/  portfolio  must  be 
presented  during  the  team  presentation.  The  outline  is  the  same  as  the  hard-copy  portfolio.  A CD  must  be  included  in  the  hard  copy  of  the  team 
portfolio.  See  Rubric.  Group  Process. 

Final  Team  Presentation:  Each  team  is  to  professionally  present  their  project,  portfolio/website  and  information  for  each 
category  on  the  presentation  outline.  This  is  a formal  presentation  where  communication  skills,  presentation  skills,  etc.  will  be 
graded.  Professional  dress  required.  An  industrial  panel  will  observe  the  presentations.  Presentation  CD  must  be  in  Portfolio. 

See  Outline/Rubric. 


IX.  Academic  Misconduct:  Refer  to  the  NIU  Judicial  Code;  Immediate  and  appropriate  actions  will  occur  for  any  students  behaving 
inappropriately,  e.g.,  cheating,  will  be  dismissed  from  the  course  immediately. 

X.  Professor’s  Role:  This  course  involves  the  professor  and  graduate  assistant  in  a variety  of  roles;  the  professor  will  provide  a scenario, 
objectives,  and  standards  and  then  guide,  coach,  and  direct  most  of  the  time,  however,  there  will  be  some  lectures.  This  course  is  performance 
based,  thus,  there  are  usually  no  traditional  objective  tests.  There  are  subjective  tests  in  the  form  of  the  5-10  minute  learning  papers,  essays  and 
the  text  project  to  determine  concept  attainment.  Students  will  construct  knowledge/skills  while  engaged  in  learning  & performances. 
Assessment  will  occur  as  learning  occurs. 

XI.  Professor’s  Notes: 

1.  Unexcused  absences  could  result  in  one  letter  grade  reduction  each  (7pts).  Class/lab/  team  meetings/work  sessions  attendance  mandatory. 
Tardiness  unacceptable.  Door  may  close  when  class  begins;  late  admittance  may  not  be  permitted  according  to  prof.’s  perogative. 
Unexcused  class/team  tardies,  I point  per  30  minutes  IF  you  are  allowed  in  and  door  is  open;  don’t  count  on  door  being  open. 

2.  The  professor  reserves  the  right  to  determine  the  final  grade  in  the  case  of  a student  who  does  not  perform  on  the  team. 

3.  Unexcused  late  projects/assignments  will  result  in  point  reduction,  2 points  per  day  late. 

4.  Dress  code:  no  hats  in  lab  ever!  Professional  dress  required  for  final  presentation. 

5.  Monitor  language  in  class/lab  at  all  times;  good  grammar  and  communication  skills  expected  at  all  times;  professional  language  expected. 

6.  Students  are  required  to  see  the  Writing  Center  tutor  for  all  written  assignments  until  approved  otherwise,  at  least  2 visits  per  assignment;  3 
visits  required  for  paper.  (I)  Meet  once  to  design  paper,  then  meet  with  draft  in  hand  (2-3)  twice  and  rewrite.  An  appointment  to  plan  the 
written  assignment  with  no  draft  for  review  would  still  require  2 other  visits  for  all  other  assignments. 

7.  Unannounced  individual  portfolio  checks  throughout  course;  5 point  penalties  for  portfolios  not  up  to  date  each  time. 

8.  No  cell  phone  ringers  in  the  class  or  lab  at  any  time;  5 points  deducted  for  in-class  interruptions.  See  professor  exception  approval. 
9.Students  cannot  pass  class  without  ALL  assignments  turned  in.  Student  will  receive  an  I (incomplete)  until  all  assignments  are  turned 
in.  Penalties  may  occur  for  grades  of  Incomplete. 

XII.  Support  Services  Available  for  Students:  The  NIU  writing  center  provides  tutoring  for  writing.  Students  in  this  class  are  required  to  use 
that  service  for  all  written  assignments;  each  writing  assignment  requires  two  visits/critiques  and  rewrites  before  assignment  can  be  handed  in  to 
professor.  Tutor  signatures  and  forms  are  required  to  be  turned  in  with  written  products.  Math  and  science  tutors  available  in  College.  NIU 
accommodations  for  any  student  with  special  needs.  See  professor  individually. 

XIII.  References  on  reference  in  Fonnders  Library  on  NIU  main  eampns:  Kerzner.  Smith.  Project  Management  & Teamwork.  McGraw-Hill.  Angns, 
Gnndersen,  Cultinane.  Planning.  Performing  and  Control-  ling  Projects.  Prentice  Hall.  2000;  Dinsmore.  Hnman  Factors  in  Project  Management. 
Dinsmore.  Project  Management.  Thomsett.  The  Little  Blaek  Book  of  Project  Management.  AMACOM;  Kerzner,  Thamhain.  Proieet  Management 
Operating  Guidelines.  VNR.;  Rosenau.  Sueeessful  Project  Management.  VNR.;  Weiss,  Wysoeki.  5-Phase  Proieet  Management.  Addison  Wesley; 
Cleland,  Gareis.  Global  Project  Management  Handbook.  McGraw-Hill.;  Miller.  Visual  Project  Planning  & Scheduling;  Barkley.  Savior.  Customer 
Driven  Proieet  Management.  McGraw-Hill;  Lewis.  Mastering  Proieet  Management.  McGraw-Hill;  Forseberg,  Mooz,  Goterman.  Visualizing  Project 
Management.  Wiley;  Dinsmore.  Winning  in  Business  With  Enterprise  Project  Management.  AMACOM;  Graham,  England.  Creating  an  Environment 
for  Successful  Projects.  JoseyBass;Gray,  Larson.  Project  Management.  McGraw  Hill.  2000.J  Cleland.  Proieet  Management.  McGraw-Hill;  Meredith, 
Mantel.  Project  Management.  Wiley.  2000;  Lewis.  Team-based  Project  Management.  AMACOM;  Kliem,  Ludin.  Project  Management  Practitioner’s 
Handbook.  AMACOM;  Kerzner.  In  Search  of  Exeellence  in  Project  Management.  ITP  VNR;  Ruskin,  Estes.  What  Every  Engineer  Should  Know 
About  Project  Management.  Dekker;  Buttrick.  The  Project  Workout.  ET  Pitman;  Rowenau.  Project  Management  for  Engineers.  Ran  Nostrand 
Reinhold;  Cleland.  Eield  Guide  to  Project  Management.  ITP  VNR;  Briner,  Geddings,  Hastings.  Project  Leadership.  Van  Nostrand  Reinhold; 
Rosenau.  Successful  Project  Mgmt.  Wi. 
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XIV.  Course  Requirements  Check  Off 

Individual  Contributions:  Benchmark=98-100  (This  means  that  you  set  the  standard  for  others.) 

(7)  Text  Project  A=93-100  points 

B=92.9-85  points 

(5)  Project  Research  C=84.9-77  points 

D=76.9-70  points 

(5)  Project  Design  F=Below  70  points 

(7)  Literature/Internet  Research  Tables  A & B 

(5)  Career  Project 

(5)  Industrial  Case  Study 

(7)  Paper 

(7)  Midterm:  Individual  Project  Plan 

A). 

(5)  Software  Workshop/Test 

(1)  Project  Feedback  Logs 

(5)  Professor’s  Overall  Assessment 

(65)  Total  Individual  Points  Possible 

Team  Contributions: 

(5)  Team  Manual 

(3)  Community/Leadership  Service  Project/ Articles 

(5)  Team  Project  Plan 

(7)  Team  Project  & Assessment  (Final  Exam) 

(5)  *Peer  Assessment  Process/Team  Success 

(5)  *Team  Member  Participation 

(5)  Team  Presentation  & Success  (Final  Exam) 

*Individual  Presentation  (in  Team  Pinal  Presentation) 

(5)  Team  Project  Portfolio  & Website 

(35)  Total  Team  Points  Possible 


IP  a teams  goal  is  to  be  a Benchmark 
Team,  where  the  team  best  exemplifies 
the  course's  highest  standards  for 
teams . . . where  the  team  sets  the  standard 
for  other  teams,  then  every  team 
member  in  that  team  must  maintain 
98%  or  ultimately  98  points  each  for  the 
course. 


Note:  To  keep  track  of  your  progress, 
add  the  possible  points  of  work  to  date; 
then  figure  the  percentage,  e.g.. 

Text  (7)+P.Research(5)+P.Design(5)=17 
.93  X 17  = 15.81  = lowest  possible  score 
or  point  value  to  maintain  an  A(lowest 

IP  your  goal  is  to  be  a Benchmark 
Student,  where  your  work  best 
exemplifies  the  (confirmed  by  Professor) 
course's  highest  standards  ...where  you 
set  the  standard,  then  you  must  maintain 
no  lower  than  98%  or  ultimately  98 
points  for  the  course. 
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Faculty  Development 
Relevant  Literature  on  Reflective  Practice 

To  review,  the  NIU  CEET  initiative  is  all  about  change.  To  implement  the  Scholarship  of 
Teaching  through  classroom  research,  there  was  a need  to  first  create  a faculty  learning 
community  and  then  provide  faculty  development  for  the  “community.”  As  described  above,  a 
needs  assessment  was  performed;  overall  goals  were  identified;  the  program  was  designed,  and 
then  both  a literature  and  program  search  were  conducted  to  determine  if  there  were  any  existing 
programs,  in  part  or  whole,  that  matched  our  needs.  We  found  no  programs  and  very  few 
components  that  could  be  pulled  together  to  create  the  program  that  we  felt  was  necessary  to 
prepare  the  faculty  community  for  the  Scholarship  of  Teaching.  The  underlying  philosophy  of 
our  program  was  that  of  engaging  faculty  members  in  a fully  integrated  program.  Teaching, 
learning,  and  student  assessment  along  with  several  other  topics  are  not  easily  separated,  as  they 
are  truly  interdependent.  We  wanted  a “program,”  not  a series  of  workshops;  we  wanted  to 
follow  our  faculty  members  into  their  classrooms  where  they  would  begin  their  experimental 
research.  That  was  also  an  important  aspect  of  the  “program”  approach  - to  include  the 
classroom  research  as  a component  of  the  faculty  development  program.  Once  the  needs  were 
clearly  identified,  the  literature  was  reviewed  and  documented,  and  finally,  the  university 
Professional  and  Organization  Development  Network  (POD),  a network  of  faculty  development 
professionals  and  units  across  universities  nationally,  was  tapped  to  see  if  any  other  university 
had  programs  important  to  know  about  or  draw  from.  We  then  developed  our  program  and 
engaged  in  the  development  of  faculty  on  the  Scholarship  of  Teaching.  See  Section  A. 3 for  the 
CEET  Vision,  Mission,  Goals,  etc. 

Our  underlying  premise  was  that  change  cannot  occur  without  serious  reflection  about  current 
practices,  the  critical  analysis  of  the  effectiveness  of  those  practices,  diagnosis  regarding  the  less 
effective  practices,  identification  of  potentially  beneficial  changes,  decisions  about  engaging  in 
new  practices,  research  on  practices  to  evaluate  their  effectiveness  on  student  learning,  using  the 
research  results,  other  feedback  and  evaluation  information  to  close  the  change  loop,  and  finally, 
to  fully  implement  new  practices.  Thus,  we  engaged  in  formal  critical  reflection  to  determine  the 
GAP  between  “current  practices,”  quality,  and  our  “vision,”  which  began  the  Reflective  Practice 
journey  for  our  learning  community  of  faculty.  We  are  informed  by  the  selected  and  relevant 
models,  thoughts,  and  faculty  development  strategies  described  below. 
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CEET  Scholarship  of  Teaching  Learning  Community  Model 


Figure  A. 5. 15: 

Global  Communities  of  Practice  and  Interest 


Regional  & National  Public  Communities  of  Interest 


Business  & Industry  Communities  of  Practice 


(Scarborough,  2006) 
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'igure  A.5.16:  Reflective  Practice:  The  Scholarship  of  Teaching  - The  CEET  Model 
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Reflective  Practice  for  Educators  - Change  through  Reflective  Practice 
To  us,  reflective  practice  by  our  faculty  members  engages  them  in  inquiry;  it  asks  faculty 
members  to  consider  their  current  reality;  develop  a vision  and  goals  about  what  they  would 
like  to  accomplish  or  change;  engage  in  learning  what  is  necessary  to  achieve  the  change 
goals;  evaluate  the  effectiveness  of  the  changes  through  research  methods  and  procedures; 
and,  finally,  implement  the  changes  that  worked  well.  And  for  those  changes  that  did  not 
result  in  the  desired  outcomes  to  begin  the  cycle  of  inquiry  again.  This  process,  however, 
requires  a kind  of  positive  tension  called  for  by  Martin  Luther  King  (1986)  and  mentioned  by 
Senge  (1990)  as  “creative  tension,”  the  energy  that  exists  in  the  gap  between  one’s  current 
reality  and  vision  - the  desired  future  state  of  being.  Reflective  practice  has  no  purpose 
without  action;  if  one  stops  with  describing  the  current  reality  without  the  action  steps  to 
move  toward  achieving  the  new  vision  (e.g.,  the  identification  of  new  goals,  learning, 
evaluating,  and  implementing),  then  that  reflection  means  very  little.  Thus,  we  have  tried  to 
provide  a faculty  development  program  to  ensure  that  a complete  cycle  of  change  can  take 
place  by  critically  examining  the  current  reality;  identifying  desired  changes;  studying  the 
effectiveness  of  those  changes  through  research  and  evaluation;  and  ultimately  determining 
what  to  fully  implement  and  what  to  study  next.  We  created  a learning  community  where 
faculty  members  can  engage  together  for  a richer  process  and  result.  In  reviewing  other 
models,  we  have  learned  that  facilitators  of  professional  development  must  first  accept  the 
following  key  assumptions  and  beliefs  about  those  participating  in  the  development 
(Osterman  & Kottkamp,  2004,  pp.  73-78). 

Credo  for  Professional  Development: 

1 . Everyone  needs  professional  growth  for  opportunities 

2.  All  professionals  want  to  improve 

3.  All  professionals  can  learn 

4.  All  professionals  are  capable  of  assuming  responsibility  for  their  own  professional 
growth  and  development 

5.  People  need  and  want  information  about  their  own  performance 

6.  Collaboration  enriches  learning  and  professional  development 

The  models  below  compare  traditional  approaches  to  more  collaborative  and  supportive 
approaches.  Each  reflects  our  goals  for  faculty  development  process  and  results. 
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Figure  A. 5. 17:  Contrasting  Approaches  to  Professional  Development 


TRADITIONAL  MODEL 


REFLECTIVE  PRACTICE 

Understanding  & Competence 


(Osterman  & Kottkamp,  2004,  p.  16) 

Figure  A.5.18:  Contrasting  Model  I and  Model  II 


Organization  ill 
Consequences 

Defensiveness 
Passivity 
Resistance 

Key  strategies 

Manage  Unilaterally 
Protect  Self  and  Others 
Be  Rational. 

Key  Assumptions 

Untlatoral  Control 
Conflict  Avoidance 


Organizational 
Consequences 
Commitment 
Problem-solving 
Creativity  & Learning 

Key  Strategies 


(Osterman&  Kottkamp,  2004,  p.  70) 


The  inquiry  strategies  recommended  by  Osterman  and  Kottkamp  (2004)  are  attending  to 
practices,  reflecting  upon  them  and  their  effectiveness,  and  then  questioning  what  could  be 
more  effective  or  considering  new  practices  that  could  have  potential.  Their  processes  and 
recommendations  fit  our  initiative  well,  especially  the  analysis  aspect  leading  to  change. 
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Figure  A. 5. 19:  Inquiry  Strategies 

1 1 INQUIRY  STRATgGIES  || 


Attending 

1 

Reflecting 

1 

Questioning 

1 

(Osterman  & Kottkamp,  2004,  p.  83) 

Sullivan  and  Glanz  (2006)  support  the  inquiry  model  of  using  problems  as  the  basis  for  the 
inquiry.  "Change  and  learning  occur  when  a problem  or  question  exists  for  which  one  sees 
[or  seeks]  a solution  or  answer”  (p.  13).  Our  model  has  the  research  and  evaluation  built  into 
it,  and  we  have  created  a faculty  learning  community.  Thus,  our  strategy  and  community 
environment  reflect  this  model  as  well.  However,  the  Sullivan  and  Glanz  model  reveals  a 
process  that  is  problem  centered,  identifying  a problem  upon  which  to  focus  the  inquiry.  We 
planned  our  approach  somewhat  differently;  as  noted  above,  our  needs  analysis  revealed  our 
strengths  and  weaknesses  and  areas  of  needed  improvement.  That  information  led  us  to 
develop  desired,  ultimately  expected,  outcomes.  Therefore,  we  designed  and  organized  the 
faculty  development  program  and  the  overall  initiative  to  achieve  the  desired  outcomes.  Our 
process  was  inquiry  driven  and  the  overall  process  reflects  that  of  the  Sullivan  and  Glanz 
model. 

Figure  A. 5. 20:  Reflective  Practice 

f igure  l.l  Ixi’llt  Tti\  i-  1‘r.Klitf  An  i ApenenlMl  I iMriuiig  C vcie 


I Krfh,  iKf  for  I .1urnr,*r:.  t liinnn 


(Sullivan  & Glanz,  2006,  p.  14) 
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Also  we  have  created  the  community  that  Gladwell  recommends.  In  agreement,  we  have 
tried  to  base  our  program,  content  and  process,  and  faculty  learning  on  research,  established 
theory,  and  best  practices  relevant  to  our  desired  outcomes,  using  the  best  and  most 
appropriate  methods  and  procedures.  We  agree  with  Sullivan  and  Glanz  that  “to  bring  about 
a fundamental  change. . .that  would  persist  and  serve  as  an  example  to  others,  you  need  to 
create  a community  around  them,  where  these  new  beliefs  could  be  practiced,  expressed,  and 
nurtured  [safely]”  (as  cited  in  Gladwell,  2000,  p.  173).  “Envisioned  change  will  not  happen 
or  will  not  be  fruitful  until  people  look  beyond  the  simplicities  of  information  and  individuals 
to  the  complexities  of  learning,  knowledge,  judgment,  communities,  organizations,  and 
institutions”  (Gladwell,  p.  213). 

Perhaps  we  can  assume  that  Problem  Identification  in  the  Sullivan  and  Glanz  model  includes 
generating  a vision,  goals,  etc.  Our  process  possibly  differed  slightly  in  that  we  first  engaged 
in  a needs  analysis  - survey  research  - to  seek  perspectives  from  students  and  professors 
about  the  “teaching”  they  were  experiencing.  This  was  the  precursor  to  the  identification  of 
problems  - or  for  us,  outcomes  to  achieve.  The  Dean  and  Lead  Professor  generated  a vision 
and  goals  for  the  college  that  the  faculty  accepted  as  a beginning  point;  then  program  leaders 
engaged  participating  faculty  members  in  current  practice  analyses,  and  followed  that  by 
identifying  teaching  and  learning  desires,  goals,  and  changes  - defined  in  our  program  and 
college  outcomes  statements.  These  became  our  “problems”  (outcomes)  for  inquiry  in  the 
model  above.  However,  rather  than  focus  so  intensely  on  problems,  although  those  were 
clearly  identified  in  the  analysis  processes  described  in  the  program  description,  we  focused 
more  on  the  positive  changes  we  would  like  to  see  occur  regardless  of  how  well  we  were 
teaching  and  students  were  learning.  Yes,  we  do  clearly  understand  our  problems,  but  the 
more  positive  approach  was  one  of  identifying  teaching  practices  that  would  increase 
learning  by  all  students.  Thus,  we  strategically  identified  outcomes  that  set  the  stage  for 
purposeful  change  and  continual  improvement  in  teaching  and  student  learning.  We  hoped 
this  focus  would  keep  us  from  getting  too  mired  in  the  “problems,”  and  instead  the  focus  on 
our  outcomes  would  keep  us  centered  and  on  what  we  wanted  to  change. 

Also  somewhat  strategically,  we  wanted  to  generate  questions  on  teaching  and  learning  that 
would  stimulate  experimental  research  in  the  classroom.  If  we  were  to  focus  on  “problems” 
from  the  perspective  that  people  did  things  wrong  all  the  time,  it  might  have  been  difficult  to 
move  forward  as  quickly  as  we  did.  Thus,  in  creating  a faculty  learning  community  where  the 
participants  could  identify  strategies,  models,  techniques,  or  procedures  on  teaching  and 
learning  that  they  did  not  use  (usually  simply  because  they  had  no  background),  choose  some 
to  try,  experiment  in  the  classroom,  execute  some  common  changes  that  led  to  course 
improvements,  and  more,  we  focused  on  how  to  improve  by  directly  addressing  a myriad  of 
problems  (our  program  outcomes)  without  making  the  faculty  members  feel  as  if  they  were 
not  professional  or  teaching  well.  By  informing  them  about  teaching  and  learning,  we 
stimulated  their  interest  and  created  the  desire  to  try  new  ideas,  strategies,  models,  and 
procedures.  We  hoped  this  would  lead  the  professors  to  become  aware  of  their  research 
potential  on  the  Scholarship  of  Teaching  and  that  they  actively  would  contribute  to  the 
research  and  bodies  of  knowledge  on  teaching  and  learning  in  higher  education. 
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When  striving  to  achieve,  in  one  pilot  initiative,  the  numerous  outcomes  we  identified  and 
the  change  process  and  program  can  be  complex.  Therefore,  we  tried  to  resolve  some  of  the 
complexity  issues  before  beginning  the  initiative.  Kotter  (1996)  informs  us  by  identifying  the 
most  common  mistakes  in  the  change  process  as  (as  cited  in  Dufour  & Baker,  1998): 

1 . Allowing  too  much  complacency 

2.  Failing  to  create  a sufficiently  powerful  guiding  coalition 

3.  Underestimating  the  power  of  vision 

4.  Under-communicating  the  vision  by  a power  of  10 

5.  Permitting  structural  and  cultural  obstacles  to  block  the  change  process 

' 6.  Failing  to  create  short-term  wins 

7.  Declaring  victory  too  soon 

8.  Neglecting  to  anchor  changes  firmly  in  the  culture  (p.  51) 

Kotter  (1996)  and  other  leaders  insist  that  successful  change  requires  a sense  of  urgency. 
Without  that,  there  will  be  minimal  response.  That  lack  of  urgency  or  slow  response  has 
occurred  regarding  the  call  for  the  reform  of  K-12,  and  it  took  many  years  for  schools  and 
communities  to  take  action.  Now  that  there  have  been  10-15  years  where  the  public  has 
called  for  the  reform  of  higher  education,  there  appears  to  be  a lack  of  sense  of  urgency. 
However,  it  is  important  to  note  that  institutions  of  higher  education  have  engaged  in  change 
for  quality  improvement  (e.g.,  improved  assessment  and  evaluation,  teaching  centers,  and 
other  changes).  Whether  these  changes  constitute  a full  reform  movement  in  consideration  of 
the  issues  regarding  the  relationship  between  teaching,  student  learning,  and  research  might 
be  a different  question,  but  from  an  industrial  perspective,  the  Kaizen  process  is  very 
effective  in  that  many  small  changes  or  improvements  lead  to  great  change  and  results  over 
time  (Goetsch  & Davis,  2006).  I believe  this  is  where  our  strategy  lies  - to  intrigue  and 
motivate  professors  to  engage  in  critical  reflection  about  their  teaching  practices  and  student 
learning;  then  to  follow  through  by  identifying  and  implementing  changes;  and  to  complete 
the  closed  loop  process  by  evaluating  the  effectiveness  of  their  changes,  resulting  in  a very 
different  teaching  and  learning  environment  where  the  scholarship  of  teaching  becomes 
important  and  recognized  as  an  exciting  and  worthy  academic  endeavor.  We  envision  the 
formal  classroom  research  called  for  when  Boyer  (1990)  and  others  defined  the  Scholarship 
of  Teaching.  Therefore,  the  change  process  we  have  begun  involves  faculty  members  in 
formal  experimental  classroom  research  as  well  as  in  formal  evaluation  of  the  faculty 
development  model  and  program. 

Change  through  Constructivism 

When  engaging  professionals  in  learning  new  knowledge  or  in  the  development  of  new 
practices,  the  process  should  be  no  different  than  how  we  ask  them  to  work  with  their 
students.  Elsewhere  in  this  document,  we  have  accepted  and  advocated  that  our  faculty 
members  engage  students  in  constructing  their  own  knowledge  through  the  learning  process. 
Professional  development  should  be  no  different;  we  should  engage  faculty  in  constructing 
their  own  knowledge  and  model  in  the  professional  development  process,  what  we  are  asking 
them  to  do  in  the  classroom.  To  review,  constructivism  means  the  individual  constructs 
knowledge  and  it  evolves  as  that  individual  evolves  through  six  cognitive  stages,  building 
knowledge  throughout  the  process  of  each  stage.  Piaget  (1971)  and  Vygotsky  (1934/1986, 
1978)  argue  for  social  constructivism  from  the  perspective  that  knowledge  is  constructed  in  a 
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socio-cultural  context  involving  cultural  tools  and  social  interaction,  which  shapes  individual 
learning  and  development.  Woolf  oik  and  Hoy  (2003)  believe  that  Vygotsky  (most  concerned 
with  the  development  of  the  individual)  bridges  the  social  and  individual  and  that  individuals, 
by  participating  in  activities  with  others,  achieve  individual  outcomes  through  working 
together  with  others.  Sullivan  and  Glanz  (2006)  agree  that  Vygotsky  bridged  the  two 
perspectives,  blending  the  social  and  individual.  The  learner  "negotiates  knowing... [and] 
stretches  just  enough  to  construct  new  knowledge  slightly  above  the  current  level  of 
knowledge"  while  engaged  with  another;  through  that  support,  the  problem  (or  change)  is 
solved  (achieved)  (p.  15).  This  has  been  identified  by  Vygotsky  as  "zone  of  proximal 
development  (ZPD)"  (Zepeda,  2000).  The  dual  concept  of  individual  and  situated  or  social 
learning  together  (learning  community)  through  reflective  practice  can  lead  professionals  to 
construct  self  knowledge  that  will  in  turn  facilitate  the  creation  of  the  [improved]  educational 
culture.  Whether  considering  student  learning  or  professional  development. 

Reflective  practice  draws  from  constructivism,  experiential  learning,  and  situated 
cognition  [in  a relevant  and  social  context] . Learning  is  an  active  process  requiring 
involvement  of  the  learner.  Knowledge  cannot  simply  be  transmitted.  For  learning  to 
take  place,  professionals  must  be  motivated  to  learn  and  have  an  active  role  in 
determining  the  direction  and  progress  of  learning. 

Meaningful  problems  engage  people  in  learning.  Learning  must  acknowledge  and 
build  on  prior  experiences  and  knowledge.  Accordingly,  professionals  need 
opportunities  to  explore,  articulate,  and  represent  their  own  ideas  and  knowledge. 
Learners  construct  knowledge  through  experience.  Opportunities  to  observe  and 
assess  actions  and  to  develop  and  test  new  ideas  facilitate  behavioral  change. 

Learning  is  more  effective  when  it  takes  place  as  a collaborative  rather  than  an 
isolated  activity  and  in  a context  relevant  to  the  learner.  (Osterman  & Kottkamp, 

2004,  p.  16) 

Therefore,  we  created  a faculty  learning  community  and  engaged  them  constructively  and 
actively  in  learning  and  development  related  to  their  individual  disciplines,  courses,  and 
students  using  traditional  assessments  beside  performances  and  the  development  of  products 
for  immediate  classroom  use  with  students.  Thus,  our  initiative  was  social/collaborative, 
directly  relevant  to  their  disciplines  and  teaching,  and  constructive  in  learning  and 
development. 

Communities  of  Practice.... Knowledge  Communities 

The  transformation  of  knowledge  within  learning  communities  requires  good  communication 
of  important  knowledge  if  it  is  to  be  revealed,  evolved  and  transformed.  Important  to  this 
process  is  an  aspect  of  group  or  team  communication.  If  not  very  careful,  and  this  very  often 
happens,  when  groups  convene  to  discuss  something,  problem  solve,  or  engage  in  decision 
making,  the  content  of  conversation  involves  the  expression  of  very  "common"  information. 
In  other  words,  the  knowledge  presented  by  individuals  is  a great  deal  of  common  knowledge 
or  knowledge  that  "everyone  knows.”  It  is  important  to  note  that  real  dialogue  versus  mere 
conversation  ("new"  information  or  information  individual  or  unique  to  each  participant  that 
others  in  the  group  do  not  know  because  of  different  backgrounds,  education,  work 
experience,  etc.  across  participants)  begins  to  surface.  That  is  when  more  important  and 
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knowledge  new  to  each  other  becomes  available  for  the  group  to  consider  and  exploit  for 
their  positive  purposes  (Goestch,  2006).  This  is  when  both  individual  learning  and  team  or 
community  learning  leaps  to  become  greater  than  the  sum  of  the  number  of  members 
involved.  As  Senge  (1990)  notes,  team  learning  is  greater  than  the  individual  learning  that 
occurs  among  team  members.  This  is  similar  to  the  interaction  effect  often  associated  with 
research.  When  two  treatments  are  applied,  there  is  also  a result  of  the  interaction  of  the  two 
treatments,  not  just  a result  of  each  treatment  individually.  With  teams  or  communities  of 
learning,  if  sharing  knowledge  known  individually  rather  than  repeating  commonly 
understood  knowledge,  there  is  first  greater  learning  by  individual  members  and  then 
exponentially  greater  learning  by  the  team  or  community. 

Our  knowledge  economy  is  often  linked  by  organizational  researchers  to  "communities  of 
practice."  These  communities  are  where  reflective  practice  lives.  Communities  of  practice  are 
described  as 

companies  [for  us  the  university]  at  the  forefront  of  the  knowledge  economy  are 
succeeding  on  the  basis  of  communities  of  practice,  whatever  they  call 
them. . .Communities  of  practice  are  groups  of  people  who  share  a concern,  a set  of 
problems,  or  a passion  about  a topic,  and  who  deepen  their  knowledge  and  expertise 
in  this  area  by  interacting  on  an  ongoing  basis.  (Wenger,  McDermott,  & Snyder, 

2002,  p.  4) 

Knowledge  community  beliefs  involve 

• Knowledge  lies  less  in  its  databases  than  in  its  people  (Fullan,  2003,  p.  121) 

• For  all  information's  independence  and  extent,  it  is  people,  in  their  communities, 
organizations,  and  institutions,  who  ultimately  decide  what  it  all  means  and  why  it 
matters  (p.  18) 

• A viable  system  must  embrace  not  just  the  technical  system,  but  also  the  social 
system  - the  people,  organizations,  and  institutions  involved  (p.  60) 

• Knowledge  is  something  we  digest  rather  than  merely  hold.  It  entails  the  knower's 
understanding  and  having  some  degree  of  commitment,  (p.  120) 

Building  a learning  community  involves  the  development  of  organizational  and  personal 
visions  articulated  through  statements  that  can  communicate  to  the  community  at  large. 

These  should  be  simple  and  brief,  although  they  are  usually  far  too  long  when  developed  by 
educators.  Those  participating  in  communities  need  to  be  trained  and/or  educated  about 
group  dynamics,  communication,  and  conflict  resolution  and  in  the  use  of  what  might  be 
perceived  as  corporate  techniques  (e.g.,  force  field  analysis,  fishbone  process,  double  reversal 
process,  rating  and  ranking,  and  using  the  pieces  of  the  pie  process,  etc.)  (Sullivan  & Glanz, 
2006). 

Applications  in  Higher  Education 

Universities  are  organizations  whose  communities  of  practice  across  its  disciplines  are  both 
disciplinary  specific  and  common.  Of  course  each  discipline  has  its  own  body(ies)  of 
knowledge  and  particular  practices  and  although  there  is  certainly  integrated  knowledge  and 
practices  across  disciplines,  most  view  individual  disciplines  as  separate  from  one  another. 
However,  the  common  knowledge  and  practices  that  should  exist  across  learning  institutions. 
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whose  goals  are  to  engage  in  research  and  also  to  teach,  are  about  research,  teaching  and 
student  learning.  Therefore,  a common  and  interdisciplinary  professional  activity  should  be 
research  about  teaching  and  student  learning.  Universities  are  complex  organizations  because 
these  institutions  and  its  professionals  bear  the  responsibility  of  contributing  to  the 
knowledge  and  best  practices  of  specific  disciplines,  but  they  also  bear  the  responsibility  of 
adding  to  the  knowledge  about  teaching  and  learning.  It  then  makes  sense  for  each  unit  to 
have  a teaching  and  learning  aspect  of  its  vision,  aligned  beside  the  vision  that  usually 
includes  disciplinary  research  and  service.  This  then  widens  the  many  disciplinary  learning 
communities  and  their  respective  communities  of  practice  to  include  a teaching  and  student 
learning  vision,  which  directly  impacts  one  aspect  of  the  community  of  practice  with  which 
they  are  engaged,  and  also  the  communities  of  practice  they  are  sending  their  students  into, 
the  employers  of  our  graduates. 

In  establishing  what  we  want  to  accomplish  with  our  faculty  learning  community  and  how 
we  want  to  go  about  it,  we  further  reviewed  the  work  of  others  in  the  hope  it  would  confirm 
or  inform  our  initiative.  Information  from  the  literature  is  woven  throughout  the  entire 
document  where  relevant.  Below,  however,  literature  is  reviewed  about  the  professional 
development  of  faculty  as  well  as  information  from  the  university  national  network. 
Professional  and  Organizational  Development  Network  (POD). 

Professional  Staff  Development  Models 

Fraser  (2005)  provides  a concept  of  development  that  addresses  both  development  of  the 
environment,  which  includes  organizational  mission,  goals,  plans,  strategies,  structures,  and 
support  systems,  as  well  as  quality  assurance  and  improvement  measures  and  the 
development  of  the  academics  and  others  who  play  a role  in  facilitation  of  learning.  This 
approach  to  developing  the  whole  environment  is  termed  "educational  development"  within 
universities.  In  defining  today's  context.  Lines  (as  cited  in  Fraser,  2005)  indicates  that  a 
paradigm  shift  had  to  occur  between  the  industrial  and  information  ages.  This  resulted  in  a 
shift  of  universities  from  a regional  to  a global  position.  Therefore,  the  work  of  academics 
became  everything  from  “teaching  on  campuses  locally,  in  foreign  countries,  to  collaborative 
research,  and  development  across  national  boundaries,  to  conducting  'virtual'  classes  with 
students  in  a variety  of  locations”  (p.  7).  Fraser  and  Lines  go  on  to  discuss  accountability  and 
the  pressure  from  the  public  on  higher  education  as  discussed  above,  the  changing  nature  of 
universities,  the  outcomes  oriented  era,  and  finally  acknowledge  that  teachers  are  still  in  front 
of  students  in  the  classrooms  lecturing  to  a group  of  students.  Courses  are  still  on  a semester 
or  quarterly  basis  for  credits;  there  is  a growing  use  of  online  courses  and  distance  learning; 
and  there  is  a growing  range  of  choices  for  learners  as  to  time,  place,  and  pace  of  learning. 
Clearly,  the  point  is  that  the  academy  that  facilitates  learning  involves  more  than  scholars  (p. 
13).  Assuring  and  improving  teaching  quality  has  become  a university  issue.  In  the  context 
that  academics  identify  first  with  their  discipline  and  second  with  their  departments, 
regardless  of  institutional  quality  issues,  the  challenge  for  educational  development  is  to  help 
academics  think  about  their  teaching  differently.  Quality  is  not  as  easily  defined  for  teaching 
and  learning;  quality  assurance  language  as  used  in  business  may  not  quite  fit.  However,  as  a 
form  of  action  research,  it  may  resonate  with  some,  while  alternate  forms  of  quality  in  terms 
of  the  language  of  scholarship  may  work  for  others.  From  there,  we  move  on  to  achieving 
commitment,  ownership,  and  trust  if  we  are  to  engage  in  ongoing  change;  there  must  be 
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commitment  at  all  levels,  faculty  to  administration,  and  beyond  any  stakeholders  who  may 
support  that  change.  "Building  change  into  university  practice,"  transformative  change,  "the 
task  of  ‘doing  quality'  needs  to  be  built  into  the  life  of  the  university  and  incorporated  in 
taken-for-granted  university  practices"  (Patrick  & Lines  as  cited  in  Fraser,  p.  35).  Resources 
need  to  be  allocated  for 

dialogical  support. . .collaborative  support  for  groups  of  staff  working  on  educational 
change  projects. . . [for  example]  adopting  new  ways  of  representing  knowledge  and 
conceiving  of  the  relation  between  teachers  and  students.  The  change  process  will 
challenge  the  epistemology,  professional  identity,  and  established  practices  of  many 
academic  teaching  staff  - in  fact,  it  involves  cultural  change  within  academic 
departments"  (p.  35;  see  also  Martin,  1999;  Prosser  & Trigwell,  1999;  Ramsden, 
1998b;  Trowler,  1998). 

The  model  these  authors  propose  uses  critical  reflection,  action  research  as  inquiry  to 
determine  needed  changes.  They  provide  for  sharing  of  experiences  across  cultures  and 
teaching  environments,  and  they  suggest  that  traditionally  used  short  term  teaching 
development  projects  or  central  activities  in  which  people  participate  have  not  been 
particularly  adept  in  accomplishing  or  supporting  transformational  change.  Patrick  and  Lines 
feel  that  a more  successful  approach  to  quality  is  needed  in  which  group  development  and 
learning  "snowball"  and  ultimately  result  in  a learning  organization  that  “creates  a coherent, 
supportive  environment  that  is  dialogical  and  open  to  experiment  and  development”  (as  cited 
in  Fraser,  2005,  p.  36;  Martin,  1999;  Senge,  1990).  These  authors  reveal  the  details  and 
issues  of  the  creation  of  their  quality  system  that  reflect  and  confirm  our  own  model. 
Chalmers  and  O'Brien  (as  cited  in  Fraser,  2005,  p.  51)  discuss  how  today's  practitioners  need 
to  have  a working  knowledge  of  the  socio-economic,  cultural,  political  and  philosophical 
context  as  points  of  reference  for  their  professional  practice.  They  present  four  overarching 
concerns  as  the  primary  foci  for  educational  development  today: 

1 . maintaining  a corporate  memory  of,  and  sustained  engagement  in,  the  issues  and 
innovations  in  teaching  in  higher  education.  Central  educational  development  units, 
or  centers  of  teaching  and  learning,  can  be  used  as  the  "site  of  corporate  memory  of 
previous  research,  innovations,  practices  as  well  as  policy  directions  in  teaching  and 
learning. . .It  also  can  encompass  the  ongoing  and  local  developments  and  teaching 
and  learning  initiatives.”  (p.55) 

2.  engaging  in  comprehensive  and  systematic  implementation  of  teaching  and  learning 
initiatives.  Central  educational  development  units  can  stimulate  relationships  between 
departments.  Currently,  some  universities  are  moving  "back"  to  the  model  of 
providing  their  own  professional  development,  as  they  are  not  satisfied  with  what  is 
offered  by  the  central  unit  and  feel  it  does  not  meet  their  specific  needs.  Conversely, 
departments  are  criticized  as  not  addressing  the  teaching  needs;  more  often,  research 
is  the  focus.  Even  when  formal  programs  of  development  are  available,  it  is  difficult 
to  get  faculty  involved  as  the  pressures  of  research,  teaching,  administration,  and 
service  are  great.  And  although  participating  in  teaching  development  programs  are 
considered  "good  to  do,"  often  faculty  do  not  find  the  time  to  participate. 
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**Our  model  is  decentralized  in  that  the  College  is  the  provider  and  only  its  faculty 
members  will  participate.  We  have  a university  office  for  professional  development; 
however,  we  had  needs  that  could  not  be  accommodated  nor  was  any  resource  support 
available  or  provided.  Therefore,  we  can  strongly  relate  to  number  2 above. 

3.  creating  and  facilitating  communities  of  learning  involved  in  the  iterative  and 
dynamic  top-down/bottom- up  engagement  and  management  of  educational  initiatives. 
Many  centralized  educational  development  units  are  participating  in  institutional 
decision  making  and  managing  requirements  for  development  and  structures  and 
strategies  for  carrying  out  the  development.  This  can  cause  problems  in  being  viewed 
as  too  top-down  and  disconnected  from  or  imposing  work  on  departments  or 
individuals.  These  issues  can  be  resolved  through  discussion  about  what  works  and 
what  is  beneficial  and  needed  and  then  by  actively  engaging  faculties,  schools,  and 
programs  in  initiatives  or  projects  that  are  responsive  to  departments'  needs  - situated 
projects  and  activities  where  the  departments  or  faculties  identify  these  projects.  The 
aim  is  to  engage  staff  in  team-  or  project-based  activities  that  will  have  an  educational 
outcome  - staff  development  and/or  student  learning. 

**  This  is  exactly  where  our  initiative  lies  in  some  respects.  We  have  an  interdisciplinary 
college  level  learning  community  comprised  of  disciplinary  learning  circles.  The 
initiative  is  collective  and  team  oriented.  We  are  fully  supporting  the  initiative,  so  that  is 
where  ours  differs.  The  NIU  educational  development  unit  decided  not  to  assist  in 
funding  our  initiative;  and  the  Provost's  Office  provided  no  funding  for  this  initiative. 

The  Dean's  Office  for  the  College  provided  approximately  $100,000  of  real  funding 
(includes  faculty  stipends),  course  release  for  the  initiative  leader,  a part  time  graduate 
assistant,  and  other  learning  materials  and  supplies  (e.g.,  books,  flash  drives,  lunches, 
etc.). 

4.  investigating,  articulating,  and  dissemination  of  scholarship  in  (and  on)  teaching, 
learning  and  education  development.  The  emerging  emphasis  on  scholarly  teaching 
and  learning  shifts  the  point  of  reference  for  development  on  teaching  and  learning 
and  for  scholarship.  University  teachers,  professors  who  teach  are  increasingly 
actively  engaged  in  the  scholarly  development  of  their  own  teaching  for  the  purpose 
of  researching  what  teaching  strategies  succeed  in  enhancing  or  increasing  student 
learning,  interest,  and  motivation.  This  enhances  the  learning  community,  including 
the  faculty  and  student  aspects  of  that  collective  community. 

**Our  faculty  engaged  in  classroom  research  and  will  disseminate  the  results.  The 
research  goes  beyond  informal  action  research;  it  is  formal  experimental  research. 
However,  it  is  action  oriented  in  that  it  will  lead  to  greater  understanding  about  what  to 
change  in  the  teaching  and  learning  environment  and  teaching  practices.  Therefore,  our 
model  incorporates  faculty  development  on  teaching  and  learning,  scholarship  through 
classroom  research,  critique  or  critical  reflection  about  practice,  and  transformative 
change,  both  individually  by  professors  and  collectively  as  a learning  community. 
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**Our  initiative  engaged  us  in  another  formal  prong  of  research,  for  we  designed  the 
initiative  to  incorporate  research  on  the  faculty  development  model  and  program  as  well. 
There  was  a strong  feedback  and  evaluation  plan  resulting  in  evidence  about  the  model, 
program,  and  process  success.  (See  Results  B.O  - B.12)  Our  model  involved  faculty 
members  in  building  a knowledge  foundation  on  teaching  and  learning;  it  supported  them 
through  the  development  of  educational  products  to  use  in  the  classroom,  as  well  as  to 
make  more  informed  instructional  decisions  about  teaching,  and  then  also  supported  them 
to  engage  in  experimental  research  in  the  classroom,  using  what  they  learned  in  the 
program.  Thus,  the  program  supports  individual  learning,  community  engagement  and 
learning,  learning  through  individual  and  community  collaborative  research,  ultimately 
resulting  in  cultural  and  environmental  changes  for  teaching  and  learning  in  the  college. 
We  will  now  move  to  sustain  the  pilot  initiative  momentum  to  sustain  continual  changes 
through  the  new  framework,  structure,  and  program. 

Patrick  and  Lines  (as  cited  in  Fraser,  2005)  have  determined  that  educational  development  is 
most  effective  if  the  central  university  is  the  unit  of  engagement  rather  than  the  more 
traditional  and  decentralized  departmental  approaches. 

**Our  model  falls  somewhere  in  between,  as  we  engaged  as  a college  unit  involving  four 
departments.  We  will  share  our  results  with  the  university  and  broader  communities.  Having 
determined  that  the  university  approach  is  best,  the  authors  also  clearly  believe  that  the 
ultimate  responsibility  for  the  quality  of  teaching  and  learning  lies  with  the  staff  who  teach 
the  students,  the  faculties  who  offer  the  programs  of  study,  and  any  support  staff.  But  Patrick 
and  Lines ’s  central  belief  is  that  for  transformative  change  in  the  quality  of  teaching  and 
learning  to  be  sustained,  it  will  be  more  effectively  achieved  through  a holistic  and  integrated 
effort  (university-wide).  We  feel  that  our  college  unit  approach  is  somewhat  holistic  and 
integrated. 

A second  belief  of  the  authors  is  that  partnerships  should  be  developed  to  engage  in  the  tasks 
of  scholarship  of  practice  while  maintaining  the  integrity  and  authenticity  of  each  discipline 
so  that  layers  of  interaction  develop  across  the  university. 

**We,  too,  believe  that  we  are  doing  this.  We  have  four  types  of  degrees,  five  programs, 
accredited  by  two  accreditation  agencies. 

Third,  Patrick  and  Lines  (as  cited  in  Fraser,  2005)  conceptualize  the  educational  development 
unit  as  a dynamic  conduit  and  agency  of  negotiation  of  what  counts  and  is  valued  as  high 
quality  teaching  and  learning  and  the  resulting  policies  thereof.  Their  concept  is  a centralized 
educational  development  unit  that  provides  ongoing  leadership  for  the  task  of  developing  and 
enhancing  the  quality  of  teaching  and  learning  in  universities.  Many  universities  now  have 
these  units  known  as  Teaching  and  Learning  Development  Offices  or  Centers  for  the 
Enhancement  of  Teaching  and  Learning.  They,  and  many  others,  offer  a conceptual  model  of 
communication  and  collaboration  as  central  to  ongoing  development  university  wide. 

**In  my  opinion,  NIU  and  many  other  institutions,  have  accomplished  joint  involvement  in 
professional  development  modules,  workshops,  seminars;  there  is  some  sharing  through 
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communication  at  such  events,  but  NIU,  and  many  others,  has  not  yet  achieved  real  internal 
partnerships  with  interest  in  the  Scholarship  of  Teaching,  practice,  or  campus-wide 
collaborative  initiatives  foeused  on  research  and/or  changes  in  teaching  and  learning. 
Therefore,  we  determined  it  best  to  seek  support  from  the  central  unit,  but  beeause  our  seope 
was  beyond  their  support,  we  moved  ahead  on  our  own.  The  O'Brien  (2004)  model  is 
presented  for  review. 

Figure  A. 5. 21:  Model  of  Collaborative  Engagement  for  Teaehing  and  Learning 
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(O’Brien,  2004  as  eited  in  Fraser,  2005,  p.  54) 


Radloff  (as  eited  in  Fraser,  2005)  presents  the  issues  involved  in  engaging  in  the 
improvement  of  teaching  and  learning: 

• how  to  motivate  staff  to  engage  with  the  task  of  continuous  quality  improvement; 

• how  to  facilitate  and  support  staff  capability  building  in  teaching  and  learning  to 
allow  them  to  engage  effeetively  in  quality  improvement  activities; 

• how  to  support  improvement  activities  as  part  of  eontinuous  quality  improvement, 
ineluding  the  resoureing  of  such  activities; 

• how  to  evaluate  the  effectiveness  of  improvement  activities; 

• how  to  recognize  and  reward  positive  outcomes; 

• how  to  support  the  Boyer  Scholarships,  especially  the  Seholarship  of  Teaching 
and  Learning;  and 

• how  to  maintain  a eyele  of  eontinuous  quality  improvement  (amongst  other 
priorities)  with  limited  resources. 


**  As  mentioned  herein,  we  managed  to  motivate  and  reward  the  participating  faculty 
members.  However,  long  term  motivation  toward  continuous  improvement  in  teaching  and 
student  learning  may  be  a different  matter.  The  faculty  participants  of  this  first  round,  our 
pilot  initiative,  were  foeused,  produetive,  dedieated,  and  stuek  with  us  regardless  of  what  we 
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asked  them  to  do  beyond  the  18+  days  of  real  time  and  the  research  semester.  But  to  sustain 
the  pilot  initiative  activities  so  that  “continuous”  quality  improvements  will  take  very  specific 
and  visible  leadership,  organization,  and  continuous  building  and  support  of  the  faculty 
learning  community.  So  although  we  accomplished  a phenomenal  list  of  outcomes  and  have 
data  to  support  the  reporting  of  those  accomplishments,  sustainability  will  now  have  to  be 
addressed  quickly  so  that  the  initiative  does  not  lose  momentum. 

**Regarding  resources,  it  is  our  intent  to  submit  proposals  to  agencies  for  funding  to  expand 
and  deepen  our  initiative,  not  only  to  others  in  the  college,  but  to  other  institutions. 

Therefore,  one  of  our  sustainability  activities  is  to  produce  a proposal  to  the  National  Science 
Foundation  and  other  agencies  for  support  to  further  the  initiative  in  several  major  ways: 

• To  provide  the  faculty  development  program  to  all  faculty  members  interested  in 
the  College 

• To  initiate  a regional  learning  community  across  universities  of  similar  interests 

• To  extend  the  initiative  technologically  by  virtual  activity  so  more  serious  sharing 
and  research  could  occur  by  members  of  the  regional  community 

• To  include  in  our  overall  program,  teaching,  and  student  learning  evaluation 
activities  additional  components  for  ongoing  evaluation  as  a result  of  what  was 
learned  through  this  program  and  initiative 

• Although  our  bylaws  already  inherently  support  and  reward  research  on  teaching 
and  learning,  we  seek  to  move  toward  adopting  standards  for  all  four  types  of 
Boyer’s  research  so  continuous  improvements  of  high  quality  can  be  more  easily 
recognized  and  rewarded. 

**Support  will  have  to  be  generated  long  term  internally;  that  can  occur  through  the 
reallocation  of  funds,  through  the  personnel  review  process,  and  through  the  Provost’s 
Office.  We  already  have  several  teaching  and  research  awards.  The  university  Presidential 
Research  and  Teaching  Awards  come  with  $20,000  and  a semester  sabbatical  in  addition  to 
usual  sabbaticals.  The  College  has  awards  aligned  with  those  that  provide  travel  to  a major 
conference  and  often  fund  special  resource  requests.  So  we  have  acknowledged  the 
importance  of  rewarding  excellence.  To  me,  however,  one  step  remains,  and  that  is  that 
anyone  who  achieves  the  quality  standards  be  able  to  receive  those  awards  upon  attaining  the 
standards.  At  the  present  time,  we  are  limited  to  six  Presidential  Awards,  three  in  each 
category,  and  one  in  each  category  at  the  college  level,  unless  there  is  a tie.  I would  rather  see 
the  number  not  limited  so  that  when  achieved,  faculty  members  are  awarded. 

**Top  level  and  visible  leadership  at  the  Dean  and  Department  Chairs’  level  is  absolutely 
critical  for  this  initiative  to  continue. 

**Finally,  the  faculty  learning  community  must  have  committed  and  very  active  faculty 
leaders  who  lead,  manage,  organize,  hold  faculty  accountable,  and  continue  to  stimulate  the 
excitement  that  is  needed  to  sustain  action  by  the  LC. 
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Radloff  goes  further  to  identify  the  capabilities  needed  for  quality  teaching  and  learning  (as 
cited  in  Fraser,  2005,  pp.  76-77;  Taylor,  2003): 

• Engagement  locally  and  globally 

• Engagement  with  peers  and  colleagues 

• Equity  and  pathways 

• Eeadership 

• Engagement  with  learners 

• Entrepreneurship 

• Designing  for  learning 

• Teaching  for  learning 

• Assessing  for  learning 

• Evaluation  of  teaching  and  learning 

• Reflective  practice  and  professional  development 

• Personal  management 

• Management  of  teaching  and  learning 

**In  response  to  Radloff  s list,  most  of  the  items  have  been  incorporated  into  our  initial 
model,  program,  or  process.  However,  one  item  is  worth  some  discussion  and  is  sometimes 
critical  to  teaching:  student  learning  and  what  is  possible  in  the  classroom,  especially  if  the 
professor  has  a desire  to  accomplish  the  teaching  and  learning  outcomes  of  our  focus.  Our 
faculty  members  are  not  what  I would  label  as  “entrepreneurs.”  Although  they  work  with 
industry  on  contracts  and  lead  research  or  development  grants,  when  it  comes  to  funding 
teaching  or  student  learning  interests,  in  my  opinion,  they  do  NOT  consider  that  their  job. 
Personally,  however,  I have  found  it  always  possible  to  get  funds  for  research  on  teaching 
and  learning,  funds  for  classroom  supplies,  materials,  and  technology  for  more  innovative 
projects.  Eor  example,  I have  long  had  a strong  relationship  with  an  engine  company.  The 
company  donates  20-30  high  powered  engines  regularly  when  needed.  We  use  them  until 
they  function  no  longer  and  try  to  respect  the  company  by  using  them  in  every  way  possible 
for  projects,  inquiry,  and  testing  to  extend  the  time  before  making  another  request.  The 
students,  professor,  and  graduate  assistants  provide  the  company  a full  report  on  our  projects 
and  results  each  semester  from  with  pictures  and  project  goals  and  results.  This  has  allowed 
me  to  keep  the  lab  fees  lower,  to  engage  students  in  complex  problem  based  learning,  and  so 
much  more.  This,  although  simple  and  only  one  example,  is  one  entrepreneurship. 

Generally,  professors  do  not  feel  that  it  is  their  job  to  seek  funds,  technology,  equipment, 
engines,  etc.  for  teaching  and  learning  purposes,  although  many  are  very  successful  for 
academic  or  contractual  research  and  development. 

**The  Dean  has  recently  employed  a full  time  staff  member  to  develop  grant  and  contract 
opportunities.  The  position  is  new  and  not  fully  evolved  in  definition.  It  is  possible  that  he 
will  include  “entrepreneurship”  for  teaching  and  learning  purposes  in  the  classroom  with 
students  as  part  of  the  individual’s  responsibilities. 

The  following  figure  reveals  the  relationships  in  Radloff’ s model.  There  is  a direct  line  to 
principles:  evidence  based  practice  obtained  through  research  generated  information. 
Otherwise  the  model  reveals  the  relationships  between  assumptions  and  strategies  for  change. 
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Figure  A. 5. 22:  Faculty-Based  Education  Development  Model 


(Radloff  as  cited  in  Fraser,  2005,  p.  81) 


Radloff  (as  cited  in  Fraser,  2005)  discusses  Teaching  Quality  Awards  where 

the  aim  is  to  recognize  and  reward  staff  who  have  made  a significant  contribution  to 
the  faculty  goals  in  teaching  and  learning;  raise  the  profile  and  status  of  teaching  and 
learning  in  the  faculty;  provide  an  opportunity  for  staff  to  share  their  knowledge  and 
skills  about  teaching  and  learning  across  the  faculty;  and  encourage  and  support  staff 
to  nominate  for  the  University  Teaching  Awards  (p.  84). 

**I  am  pleased  to  say  that  both  the  College  and  NIU  have  such  awards,  as  mentioned  above. 
And  beyond  that,  the  faculty  participating  in  this  initiative  will  receive  monetary  awards  to 
recognize  the  great  amount  of  time  they  engaged  individually  and  together  to  achieve  the 
initiative  goals  and  formal  products,  research,  and  dissemination  activities  they  committed  to 
and  have  accomplished.  We  cannot  begin  to  pay  them  for  their  level  of  effort  and  time,  but 
what  we  have  been  able  to  do  is  to  offer  a significant  stipend,  to  use  some  regular  weekdays 
during  the  semester  (in  addition  to  some  after  summer  begins),  and  to  provide  important 
resources  such  as  books  and  other  supplies.  The  professors  received  a certificate  of 
completion  for  the  program,  and  some  of  them  received  special  resources  to  support  learning 
activities  during  the  research  semester.  For  example,  two  faculty  members  needed  the  Kolb 
Learning  Style  Inventory  for  all  their  students;  the  Dean  purchased  them.  He  also  arranged 
special  learning  space  requests  and  more.  The  support  received  was  more  than  reasonable, 
and  recognized  their  commitment  to  the  endeavor,  however,  less  than  what  our  faculty  can 
earn  as  consultants  during  the  summer  or  make  on  their  grants.  As  they  noted,  they  are  not  in 
this  for  the  monetary  rewards;  they  see  the  value  in  the  program  and  research  to  follow,  and  it 
seems  they  are  genuinely  interested  in  the  scholarship  of  teaching.  However,  although  we 
had  teaching  awards  established  before  our  initiative,  there  has  not  really  any  vehicle  for 
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sharing  knowledge  and  skills  across  departmental  faculties,  and  even  at  this  point  across  our 
initiative,  so  the  sustainability  of  that  process  has  yet  to  be  fully  structured.  The  “internal 
dissemination”  will  begin  to  occur  with  presentations  by  initiative  participants  during  fall 
semester  2007.  We  are  hoping  that  the  formalized  LC  structure  will  be  revised  and  scheduled 
by  the  community  before  Fall  2007  to  continue  learning,  research,  and  sharing  activities.  We 
are  hoping  that  they  will  all  participate  in  seeking  grants  together.  This  will  be  strongly 
supported  by  the  Dean. 

Evaluation  occurred  in  two  ways,  as  we  have  a two  pronged  research  effort  involved  in  this 
initiative.  We  researched  the  value  and  effectiveness  of  the  professional  development  model, 
program,  and  processes,  and  each  participating  faculty  member  researched  the  effectiveness 
of  selected  teaching  strategies  and  new  student  assessments  on  student  learning.  Therefore, 
there  are  layers  of  evaluation  embedded  within  our  model  that  are  more  thoroughly  discussed 
in  the  research  section.  Radloff  (as  cited  in  Fraser,  2005,  p.  85)  provides  a list  of  what  he 
(and  his  colleagues)  feels  would  constitute  useful  evidence  for  determining  the  effectiveness 
of  any  educational  development  effort.  They  use  Kirkpatrick's  (1998)  approach,  a four-step 
strategy:  satisfaction,  learning,  application,  and  impact  and  then  mapped  out  what,  for  them, 
would  constitute  useful  evidence: 

• positive  staff  reaction  to  support  and  professional  development  opportunities 
in  the  form  of  feedback  surveys,  information  comments,  participation  in 
activities  and  return  business  (satisfaction); 

• increased  staff  confidence  and  competence  in  teaching  and  learning  through 
self,  peer  and  student  evaluation  (learning); 

• improved  teaching  and  learning  practices  at  program  and  course  (subject) 
levels,  as  evidenced  in  the  design  of  learning  activities  and  assessment  tasks, 
self-reflection  and  peer  review  (application); 

• improving  student  performance,  as  evidenced  by  reduced  attrition  rates, 
increased  progression  rates,  and  enhanced  learning  outcomes  (impact); 

• increasing  student,  graduate  and  employer  satisfaction  with  learning  and 
learning  outcomes  (impact);  and 

• improving  program  performance  in  terms  of  the  university's  “business  rules,” 
namely  quality  (student  and  graduate  satisfaction),  relevance  (employability 
of  graduates)  and  viability  (demand  for  the  program  and  cost-effectiveness) 

**Our  evaluation  results  indicated  that  the  professors  were  extremely  pleased  with  the 
model,  program  content,  and  process.  They  indicated  that  it  would  not  be  reasonable  for  most 
faculty  members  to  commit  to  the  number  of  days  of  faculty  development.  However,  now 
that  we  have  piloted  the  program,  it  is  possible  to  condense  some  of  the  activities  without 
losing  quality.  Also  all  of  the  materials  are  now  prepared  for  the  fundamental  program 
components.  These  will  need  updating  but  not  total  redevelopment  for  another  program 
offering.  Other  new  components  may  be  developed  as  the  faculty  members  ask  for  some 
additional  topics  to  be  available  later  (e.g.  more  time  on  cooperative  learning,  student 
teaming,  and  classroom  and  conflict  management).  The  cooperative  learning  component  is 
already  developed  as  well  as  a very  thorough  program  on  student  teaming. 
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**We  have  set  the  stage  for  ongoing  evaluation;  it  is  more  a matter  of  what  direction  the  LC 
now  takes  as  faculty  members  begin  to  lead  the  LC.  The  leader  has  been  a faculty  member, 
but  she  has  retired;  therefore,  other  members  will  take  on  the  leadership.  Several  are 
interested. 

**We  have  set  the  stage  for  long  term  and  continuous  teaching,  assessment,  student  learning, 
etc.  changes.  The  participating  faculty  members  have  a tool  box  of  resources  at  immediate 
hand,  each  individually,  for  this  purpose.  Not  only  do  they  have  what  they  have  already 
developed,  but  they  have  the  knowledge,  tools,  and  additional  information  to  support  their 
long  term  and  continued  growth  and  changes. 

**Initiative  sustainability  is  critical  and  is  the  only  way  to  provide  evidence  of  several  of 
Radloff  s listed  types  of  evidence:  employer  satisfaction,  increased  student  learning; 
improved  program  performance,  and  graduate  satisfaction. 

Although  a much  earlier  model,  Zuber-Skerritt  (1992)  offers  a model  that  integrates 
educational  theory  and  teaching  practice  through  action  research;  our  model  does  this  as  well. 
He  defines  action  research  as  GRASP,  where  “critical  (and  self-critical)  collaborative  inquiry 
by  reflective  practitioners  being  accountable  and  making  the  results  of  the  enquiry  public, 
self  evaluating  their  practice  and  engaged  in  participative  problem-solving  and  continuing 
professional  development  (p.  19).  Each  component  is  described  in  the  table  below. 

Table  A. 5. 18:  The  Case  in  Relation  to  the  GRASP  Model 
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(Zuber-Skerritt,  1992,  p.  20) 

Also  reflecting  our  initiative  as  well  as  informing  it,  Zuber-Skerritt  (1992)  provides  a 
theoretical  framework  or  meta-theory  grounded  in  other  previously  unrelated  theories  on 
teaching  and  learning,  etc.  He  establishes  the  gap  between  theory  and  practice,  acknowledges 
the  vast  amount  of  research  on  student  learning  and  teaching  methods,  and  further  admits  that 
literature  has  had  seemingly  little  impact  on  the  practice  of  teaching  because  most  professors 
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have  not  had  any  professional  preparation  for  teaching  and  normally  do  not  access  all  the 
books,  literature,  research  on  the  topic  as  it  seems  irrelevant  to  their  endeavors.  They  are  not 
comfortable  with  the  jargon  and  terminology  used  in  educational/psychology  and  feel  it 
somewhat  pretentious. 

**This,  in  fact,  was  the  reaction  by  at  least  one  or  more  or  our  participating  professors.  The 
comment(s)  went  something  like  "so  many  terms  that  have  the  same  meanings,  etc.,"  when  in 
fact  there  were  subtle  differences  that  were  not  easily  grasped  by  those  who  had  not  been 
exposed  to  the  evolution  of  concepts  or  terms  such  as  learning  objectives  or  learning 
outcomes  or  behavioral  objectives,  etc. 

Zuber-Skerritt’s  (1992)  model  is  presented  more  fully  below,  and  although  different  as  a 
visual  or  representational  graphic  and  more  broadly  presented,  it  reinforces  our  direction.  His 
book  is  outlined  in  the  first  graphic;  the  models  reflect  various  authors’  perspectives.  Each  of 
these  serves  to  confirm  and  inform  our  direction,  model,  and  processes. 

Figure  A. 5. 23:  The  Structure  of  This  Book  [as  a professional  development  model] 


(Zuber-Skerritt,  1992,  p.  20) 
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**  Our  comment. 

Part  1:  This  model  (Figure  above)  component  addresses  the  basic  assumptions  about 
the  connection  between  theory  and  practice:  that  relationship  is  perceived  more  as  a 
dialectical  relationship  than  a dichotomy.  Practical  reasoning  is  the  bridge  over  the 
gap  between  theory  and  practice.  Zuber-Skerritt’s  (1992)  aim  is  to  show  the  relevance 
of  the  theories  to  practical  and  emancipatory  action  research  for  change.  His  action 
research  spiral  is  to  Plan,  Act,  Observe,  and  Reflect,  not  unlike  the  Deming’s  Quality 
Circle  (1956  - industrial),  plan,  do,  act,  check,  or  the  revised  one,  plan,  do,  check,  act, 
analyze  (Goetsch,  2006,  p.  19). 

Part  2:  This  component  provides  the  theoretical  basis  that  presents  the  opportunity 
and  supports  Zuber-Skerritt’s  (1992)  view  of  how  to  improve  teaching  and  learning  in 
higher  education.  This  component  is  the  foundation  for  the  model  structure  of  staff 
development  and  student  learning. 

Part  3:  This  component  serves  to  integrate  theory  and  practice  such  that  one  supports 
the  other.  It  then  drives  the  fourth  component,  faculty  development.  However,  this 
linear  approach  might  be  questioned  as  to  which  comes  first,  faculty  development  or 
research.  With  professors  who  have  no  background  in  teaching  and  learning,  it  would 
be  difficult  to  integrate  the  theory  and  practice  without  first  engaging  them  in  faculty 
development.  Perhaps  this  would  be  necessary  only  initially. 

Part  4:  This  component  reveals  the  focus  on  faculty  development,  but  we  found  that 
faculty  development  was  needed  to  reach  the  initial  point  to  be  able  to  integrate 
theory  and  practice.  There  is  a need  with  those  who  have  no  background  in  teaching 
and  learning  to  achieve  some  fundamental  understanding  of  the  theory  before  being 
capable  of  research.  Also  educational  research  is  quite  different  in  nature  than  the 
engineering  and  technology  type  of  research  our  faculty  are  used  to  performing. 

** Although  an  older  source  on  faculty  development,  Zuber-Skerritt’s  (1992)  model  certainly 
relates,  reflects,  and  supports  our  initiative.  Program  content,  process,  and  the  overall  model 
are  reflective  of  his  support  theories,  model  for  taking  action,  methods  and  strategies  for 
professional  development,  and  more. 

Esland  (1971)  also  reflected  our  faculty  development  goals,  as  well  as  what  we  hope  will 
happen  in  classrooms  with  students  (as  cited  in  Zuber-Skerritt,  1992).  Although  somewhat 
generic,  the  focus  on  learners  being  “active”  and  “negotiators  of  meaning,”  where  one 
engages  in  the  “construction  of  knowledge  and  experience,”  etc.  resonates  with  the  theory 
supporting  our  operation  premises.  (See  Literature  section  A. 6) 
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Table  A. 5. 19:  The  View  of  Humankind  in  the  Traditional  and  Dialectic  Epistomology 


Traditiunal  cpisU'roulogy 


[)iak*ctic  cpislemnlugy 


liiimdnkini.1 


Passive  receiver  of  btowicdge 


Active  seeker  and  negotiator  ot 
meaning 


Woild-prodiiCed 

Having  a static,  analytic  conception  of 
knowledge 


World  producer  and  social  piiuluci 
Heing  involved  in  an  active  construction 
of  knowledge  and  experience 


Believing  in  tnilh  and  validity  of 
knowledge 


Believing  m changing  foniis  and 
content  knowledge,  open  to  sociological 
revitalisation 


Regarding  teaching  as  the  acquisition  of 
skills  atid  techniques  to  irunsrei 
knowledge  from  teacher  to  student 


Regarding  teaching  as  active  knowledge 
and  rcnective  uiidefsi;inding  of 
cuiriculum  pedagogy  and  evaluation 


(Esland,  1971  as  cited  in  Zuber-Skerritt,  1992,  p.  35) 

The  elements  of  Mumford’s  (1990)  and  Bond  and  Pascoes  (1978)  (as  cited  in  Zuber-Skerritt, 
1992)  models  below  are  directly  in  line  with  our  goals  and  outcomes  for  this  initiative.  The 
learning  environment  reflected  in  the  Mumford  model,  especially  where  relationships  are 
represented  and  where  the  process  of  planning,  experiencing,  reviewing,  and  concluding  are 
the  context  within  which  the  relationships  function  through  a wide  range  of  activities, 
certainly  also  resonate  with  our  model,  program,  outcomes,  and  results.  The  process 
represented  in  the  Bond  and  Pascoe  model,  where  individuals  experience,  observe  and 
reflect,  form  abstract  concepts  and  generalizations,  and  test  in  new  situations  follows  the 
lines  of  action  research  or,  as  in  our  case,  formal  experimental  research.  The  teaching  cycle 
phase  seems  to  represent  informal  or  action  research  more  than  more  highly  formalized 
experimental  research.  However,  we  want  to  stimulate  our  professors  to  understand  that  both 
informal  and  formal  and  both  action  research  without  controls  or  experimental  research  with 
control  groups  are  equally  valuable  for  testing  new  ideas,  theories,  strategies,  etc.  in  the 
classroom  about  teaching  and  learning.  We  would  hope  that  the  professors  engage 
individually  and  together  using  both  strategies  and  a wide  variety  of  methods  and  procedures. 
We  can  directly  relate  to  these  models,  as  they  confirmed  our  operational  premises  and 
actions 
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Figure  A. 5. 24:  Interaction  in  Learning 


(Mumford,  1990  as  cited  in  Zuber-Skerritt,  1992,  p.  47) 

Figure  A. 5. 25:  An  Experiential  Learning  and  Teaching  Model 


(Bond  & Pascoe,  1978  as  cited  in  Zuber-Skerritt,  1992,  p.  48) 

Faculty  Development  Standards 

Dufour  and  Laker  (1998)  cite  the  National  Staff  Development  Council  for  professional 
development  standards.  The  standards  are  organized  into  three  categories:  content,  process. 
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and  context.  Content  is  the  "what,"  aetual  knowledge  or  skills  needed  by  edueators.  Proeess 
is  the  "how"  of  staff  development  or  the  means  by  whieh  educators  acquire  knowledge  and 
skills.  Finally,  eontext  refers  to  the  "organization,  system,  or  eulture"  supporting  the  staff 
development  initiatives  (NSDC,  1995,  p.  256).  In  addition,  the  Foundation  for  the 
Improvement  for  Edueation  (1996)  defines  elements  of  effeetive  professional  development: 

• identifies  that  student  learning  be  the  primary  foeus; 

• fosters  the  deepening  of  subjeet  matter  knowledge  and  greater  understanding 
about  learning  along  with  greater  appreciation  of  students'  needs; 

• assists  teaehers  to  learn  to  work  with  diverse  students  and  student  eapabilities, 
providing  the  time  for  inquiry,  refleetion,  and  mentoring; 

• requires  that  staff  development  is  rigorous,  sustained,  and  results  in  ehanged 
praetiee; 

• ineorporates  best  prineiples  and  praetiees  for  adult  learning; 

• involves  shared  deeision  making; 

• makes  best  use  of  teehnologies;  and 

• has  a vision  for  students,  (pp.  260-261) 

Although  foeused  on  K-12  professional  development,  these  standards  ean  inform  us  about 
our  own  staff  development,  for  they  are  appropriate  regardless  of  the  level  of  teaching. 

The  authors  above  agree  with  Sparks  and  Hirsh  (1997)  that 

in  a logieal  progression,  results-driven  edueation  for  students  requires  results-driven 
staff  development  for  educators. . .Staff  development's  sueeess  will  be  judged  not  by 
how  many  teachers  and  administrators  [professors]  participate  in  staff  development 
programs  or  how  they  pereeive  its  value,  but  by  whether  it  alters  instruetional 
behavior  in  a way  that  benefits  students,  (p.  5) 

**The  above  thoughts  about  staff  development  programs  and  whether  they  result  in  ehange 
in  instruetional  methods,  strategies,  models,  or  behaviors  express  our  very  real  eoneem  as 
well.  We  were  determined  NOT  to  engage  faeulty  partieipants  in  the  initiative  UNLESS  we 
aetually  ineluded  the  elassroom  component  of  the  experimental  researeh  semester.  There  are 
two  very  real  and  extreme  differenees  about  our  initiative:  (1)  the  faculty  development  was  a 
fully  integrated  “program,”  not  a series  of  workshops,  where  faeulty  members  aetually 
produeed  real  produets  to  use  in  the  elassroom,  where  they  made  new  instruetional  deeisions, 
and  where  they  eompletely  redeveloped  a eourse  to  reflect  what  they  had  learned  and  planned 
on  doing  with  students  that  was  new.  Therefore,  they  did  not  attend  a series  of  workshops 
and  then  go  home  to  do  the  work  of  developing  produets  or  ineorporating  ehange.  They  did  it 
WITH  us  while  engaged  in  the  program’s  activities.  (2)  The  other  major  difference  is  that  the 
program  INCLUDED  the  in-elassroom  researeh  semester  as  a program  eomponent.  There 
were  aetivities  that  eontinued  with  the  professors  while  they  were  teaehing  that  semester  and 
implementing  new  strategies,  using  their  new  produets,  and  more.  Therefore,  we  were 
assured  that  at  least  for  that  one  semester,  the  professors  DID  ehange  their  instructional 
practices,  used  new  educational  products,  engaged  in  experimental  researeh,  produced 
researeh  manuseripts,  and  ehanged  their  instruetional  behaviors,  models,  proeess,  and 
strategies.  We  are  trying  to  preserve  that  level  of  aetivity  and  momentum  by  eontinuing  the 


88 


faculty  learning  community  with  support.  That,  however,  cannot  be  fully  discussed  at  this 
point.  Our  initiative  now  faces  the  true  test: 

1 . Will  this  pilot  initiative  result  in  support  for  an  expanded  faculty  learning  community 
to  engage  formally  together  in  a second  round  of  the  faculty  development  program? 

2.  Will  this  pilot  initiative  result  in  support  for  the  initial  and  an  expanded  faculty 
learning  community  to  engage  formally  together  in  a second  round  of  research  in  the 
classroom  - continuing  the  Scholarship  of  Teaching? 

3.  Will  the  culture  begin  to  reflect  the  changes  stimulated  by  the  initial  program  and 
research? 

4.  Will  the  college  learning  environment  begin  to  reflect  the  preferred  teaching 
strategies,  models,  styles,  best  practices  across  more  classrooms  so  that  students  grow 
accustomed  and  expect  excellence  in  teaching  and  to  be  involved  in  research  on 
teaching  and  learning? 

5.  Will  the  structure  and  leadership  be  put  into  place  to  sustain  ongoing  development, 
growth,  and  change  by  faculty  and  administrators? 

These  questions  cannot  be  answered  today.  It  truly  is  a matter  of  leadership. 

Goleman  (1995),  along  with  many  others  (e.g..  Cooper  (1997)  on  Emotional  Intelligence  or 
EQ),  focuses  on  the  development  of  "group  IQ,"  where  a group  can  be  no  “smarter”  than  the 
sum  total  of  the  strengths  of  individual  members,  but  it  can  be  “dumber”  if  its  internal 
workings  do  not  allow  members  to  share  their  talents”  (p.  23).  This  resonates  with  Senge  and 
Goetsch  above;  the  group  can  be  smarter  than  the  number  of  individuals  if  an  “interaction” 
effect  is  occurring,  where  the  knowledge  brought  to  the  table  by  each  member  is  unlike  that 
of  other  members,  not  common,  and  if  the  process  is  transforming  - that  the  combination  of 
that  individually  different  knowledge  leads  to  or  results  in  something  greater  than  each  part 
or  the  sum  of  the  parts. 

These  authors,  especially  Goleman  (1995),  express  thoughts  about  content  for  professional 
development  of  a learning  community:  it  must  be  research  based  in  content,  focused  on  both 
generic  and  discipline-specific  teaching  skills,  and  must  expand  the  repertoire  of  learning 
strategies  to  meet  the  needs  of  diverse  learners.  Their  process  is  attentive  to  the  tenets  of 
good  teaching,  provides  coaching  to  master  new  skills,  involves  reflection  and  dialogue,  and 
is  sustained  over  time.  The  context  is  clearly  defined  as  their  school  - ours  is  the  college;  that 
the  learning  context  is  "on  the  job"  and  not  arbitrarily  scheduled  and  removed  from  their 
work  place  - the  classroom;  that  time  at  work  encourages  ongoing  development;  and  that 
time  at  work  helps  to  develop  skills.  Teaming  is  embedded  in  work.  Action  research  in  the 
classroom  is  consistent  with  their  model  and  with  what  they  are  suggesting.  We  engaged  in  a 
version  of  that  as  well  as  formal  experimental,  pre-experimental,  or  quasi-experimental 
research. 

Einally,  an  email  was  sent  out  by  the  Director  of  NIU’s  Eaculty  Development  Office  to  the 
Professional  and  Organizational  Development  Network  (POD)  across  universities 
internationally.  He  posted  a request  to  the  POD  members  asking  them  for  information  on 
faculty  development,  programs,  courses,  workshops,  etc.  He  felt  that  if  they  had  anything  to 
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offer,  there  would  have  been  responses  and  mentioned  that  the  whole  group  is  very 
responsive  to  requests  from  each  other.  There  were  four  responses  revealing  that 

• Colorado  Community  College  has  an  online  course  for  which  completion  results  in 
various  competencies  for  online  courses:  understanding  when  to  use  online 
discussion,  being  able  to  articulate  one’s  philosophy,  being  able  to  write  questions 
that  illicit  critical  thinking,  setting  criteria  for  grading,  knowing  how  to  use  small 
groups  effectively,  engaging  students  more  effectively,  etc.  Although  this  course 
relates  to  our  focus,  the  depth,  level  of  engagement,  requirement  of  products,  level  of 
intensity,  and  expected  results  are  very  different.  Their  online  Teaching  Practices  and 
Techniques  do  reflect  some  of  what  we  have  tried  to  accomplish 
(http://www.ccconline.org/FacultvSC/Training/Course  Descriptions.htm.). 

• The  Centre  for  Professional  Development  at  Seneca  College  in  Toronto,  Canada, 
offers  three  courses  as  part  of  its  probationary  program  in  which  any  faculty  member, 
whether  probationary  or  not,  could  participate.  The  program  is  also  competency  based 
to  help  faculty  develop  and/or  demonstrate  their  competencies  in  teaching  as  well  as 
planning  and  tracking.  The  program  focus  is  on  Foundations  of  Teaching  and 
Learning;  Technology-Enhanced  Learning;  and  Language  and  Cultural  Diversity.  The 
expected  competencies  are  itemized  and  related  to  what  we  are  trying  to  accomplish 
with  our  faculty.  It  engages  the  faculty  in  120  hours  of  training  across  the  three  areas 
of  focus.  Laculty  members  develop  and  submit  portfolios  to  demonstrate  previous 
experience,  and  they  may  also  submit  other  learning  opportunities  or  courses,  etc.  as 
evidence  of  competence.  Although  related  to  our  initiative,  this  is  also  very  different. 
One  important  difference  is  that  The  Centre  sets  up  mentors  for  probationary  faculty 
members,  which  is  important  for  us  to  consider.  The  goals  of  the  teaching  and 
learning  reflect  many  of  ours,  and  the  expected  educational  products  are  similar. 
However,  The  Centre’s  process  seems  very  different.  We  do  not  have  a focus  on 
technology  enhancement;  however,  we  did  have  an  inherent  focus  on  culture  and 
diversity  (dominque.giguere@senecac.on.ca  or  www.senecac.on.ca/cpd). 

• Kansas  State  University  has  an  actual  “Principles  of  College  Teaching,”  a graduate 
level  course  that  is  offered  both  to  graduate  students  intending  to  become  faculty 
members  and  to  faculty  members.  The  course  begins  with  a focus  on  learning  and 
moves  to  strategies  for  teaching.  The  class  discussions  are  very  interactive... Class 
assignments  and  projects  are  individualized  so  that  students  can  adapt  those 
experiences  to  their  current  and  future  situations.  The  students  develop  student 
learning  outcomes,  assessment  strategies,  teaching  strategies,  and  teaching 
philosophies.  They  observe  video  of  at  least  one  of  their  own  teaching  efforts. . .also 
contract  to  complete  a semester  project  related  to  teaching  and  learning  at  the  college 
level.  At  the  end  of  the  semester,  they  must  share  with  me  what  they  have  learned 
form  their  projects  (Clegg,  2005,  email),  (vclegg@ksu.edu)  The  course  objectives  and 
content  relate  to  ours  very  well;  however,  our  process  and  products  are  very  different. 

• The  University  of  South  Llorida  responded  that  it  has  a graduate  level  course 
“Seminar  in  College  Teaching”  serving  students  in  the  Ed.D.  or  Ph.D.  programs  in 
Higher  Education  and  the  M.A.  in  College  Teaching  Program,  students  in  other 
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programs,  and  recently  employed  full-time  faculty  at  two  community  colleges. 
However,  he  mentioned  that  he  “had  not  thought  to  advertise  it  to  USF  faculty  though 
your  [our]  inquiry  prompts  me  to  once  again  revisit  this  possibility.”  The  goals  and 
objectives  of  the  course  relate  to  ours,  but  once  again,  the  process  and  expected 
products  differ  greatly,  (ieison@tempest.coedu.usf.edu) 

There  are  a myriad  of  courses,  seminars,  and  workshops  being  offered  on  university 
campuses  as  faculty  development  opportunities.  We  all  share  common  national  speakers  who 
are  brought  to  campus;  we  also  all  have  various  locally-offered  workshops,  etc.  However, 
there  do  not  seem  to  be  any  “programs”  or  fully  integrated  courses  where  a faculty  learning 
community  is  created  and  the  program  engages  them  in  the  content,  process,  performances, 
and  product  development,  followed  by  experimental  research  in  the  classroom,  as  in  ours.  So 
we  offer  you  our  model,  program,  and  process  as  well  as  the  results  of  the  classroom 
research.  We  completely  reveal  all  evaluation  and  feedback  as  well.  It  is  our  hope  that  in 
doing  so,  anyone  reading  about  our  initiative  will  realize  both  its  strengths  and  weaknesses, 
and  for  those  trying  similar  initiatives,  they  can  learn  from  ours  by  testing  their  assumptions, 
possibly  reducing  the  preparation,  reducing  the  learning  curves,  and/or  benefiting  from  what 
we  have  learned  and  have  to  offer.  (See  Results  in  B.0-B.12) 


For  review,  we  have  repeated  the  program  calendar  presented  earlier,  but  have  added  a 
calendar  from  the  perspective  of  October  2005  to  May  2007.  See  calendars  below. 
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Table  A.5.20:  CEET  Initiative  on  Teaching  & Learning  Spring  Semester  Schedule  (2006)  (*assessment) 


Thursday,  Feb.  2 

Thursday,  Feb.  9 

Thursday,  Feb.  16 

Thursday,  March  2 

Thursday,  March  23 

Orientation  - Presentation 

Course  Analvsis  - Presentation 

Course  Analvsis  (cont.) 

Course  Analvsis  (cont.) 

Course  Develonment 

Faculty  Roles,  Respons.,Duty 

Knowledge  Content  Outlines 

Objectives  & Outcomes  matched 

Teaching  Models 

ABET/TAC/NAIT 

The  Scholarship  of  Teaching  - 

The 

Knowledge  Priorities 

to  assessments 

Teaching  Styles 

Standards  - Student 

National  Call  for  Action 

Embedded  General 

Learning  Styles 

Outcomes 

Action  Research 

Education  Goals 

Assessments  by  Bloom  and  Dale 

Instructional  Design 

Outcomes  by  Bloom’s 

Learning  Communities 

Student  Learning 

Critical  Thinking 

Analysis 

Knowledge  Communities 

Objectives  and  Outcomes 

Course  Calendar 

Communities  of  Practice 

Teacher,  Knowledge, 

Double  Loop  Learning 

“Reversed”  Instructional 

Assessment,  or  Learner 

Introduction 

Teaching  Professionals 

Design  Model  - Intentional 

Centered? 

Complete 

Syllabus  Development 

What  is  Learning? 

Design 

GAPS  Analysis  Summary 

Super  Syllabus 

What  is  Learning  Pedagogy? 

Test  Analvsis  - Presentation 

Taxonomies  of  Learning 

Purpose  of  Test  Analysis 

Self-Assessment:  The  First  Step  in 

Dale’s  Cone  of  Learning 

Item  Analysis 

Active  Learning 

Reflective  Practice 

Item  Difficulty 

Problem-based  Learning 

Objectives  & Outcomes 

Item  Discrimination 

Growing  up  Digital 

Program  Description  & Model 

By  Bloom’s  Taxonomy  and 

Case  Test  Analysis 

Student  SWOTs  Analysis 

Dale’s  Cone 

Flagged  Items 

Syllabus  Analysis 

Analysis  of  Results 

10/11/05-  Commitment  Mtg. 

Validity 

-Program  Description 

Reliability 

-Requirements 

Standard  Error  of 

-Expected  Outcomes 

Measurement 

-Rewards 

Using  nil’s  Testing  Services 

* Student  Learning 

*LC  Assessment  (a) 

Objectives  -Outcomes 

*Self-Competency  (a) 

*SLO  Assessment  (a) 

*Test  Analysis  Assessment  (a) 

Asse-sment  - SLO  (a) 

Thursday,  March  30 

Thursday,  April  6 

Thursday,  April  20 

Thursday,  April  27 

Break  April  28-May  14 

Test  Devel-nment  - Presentation 

Test  Develonment 

Test  Develonment 

Test  Develonment 

See  you  in  May! 

Discrete,  Objective  Items 

Test  Items/Bloom’s  Taxonomy 

Item  Writing 

Item  Writing 

Midterm  Exam  Test  Assembly 

Case  Test 

Valid  Test  Items 

final  Exam  Test  Assembly 

Constructing  Multiple  C.  Items 

Test  Development 

Constructing  Short  Answer  It. 

Develop  Items-submit 

*Test  Development 

*Test  Analysis  Re-assessment  (b) 

Reassessment  (b) 

*Test  Dev.  Assessment  (a) 
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Table  A.5.21:  CEET  Initiative  on  Teaching  & Learning  May  15-25  Schedule  (*assessment) 


Mon.,  MarchlS-,  9am-5pm 

Tues.,  March  16, 9am-5pm 

Wed.,  March,  17,  9am-5pm 

Thurs.  March,  18  9am-5pm 

Fri.,  Mar.  19 

Regroun  Dav  -Test  Analysis, 

Performance  Assessment  & 

Reflectiye  Practitioners  - 

Balanced  Assessment 

9am-5pm 

Develonment,  Assembly  Reyiew 

Rubrics  - Presentation* 

Presentation 

Course  Assessment  Plan 

Teaching  Models 

SLO/Test  Item/Bloom  Analysis 

Performance  Assessment  & 

Deyelopment  PAR  #2 

PAR  deyelopment  - finalize  Perform 

ON  OWN 

Rubric  (PAR) 

PAR  Bloom  Analysis 

Continue  SLO  actiyities 

Deyelopment  PAR  #1 

Share  and  Critique  with  Peers 

Other  Assessm.  types 

Reading 

Assignment 

TEACHING  PORTFOLIO 

Deyelopment  PAR  #3 

Map  Assessments  (Kuhs  et  al.) 

Portfolio  Organization 

Share  and  Critique  with  Peers 

Share  and  Critique  with  Peers 

Identify-  types  of  assessment  to  include 

THE  CITE  TOOL  BOX 
Tool  Box  Organization 

*See  PA  Chapter  & Ref. 

in  Course  A.  Plan 

Learning 

Actiyity 

Each  Professor  builds  his/her 

Materials 

Deyelon  descriptions,  products  for  each 

own  Tool  Box 

*See  PA  Chapter  & Reference 
Materials 

type 

*TM  Assessm. 
Product 

*TD/A  Re-assessment  (c) 

*SLO  Reassessment  (b) 

*PAR  Assessment  - Products 

Explain  Friday’s  Assignment 

Monday,  M22  9am-5pm 

Tuesday,  M23  9am-5pm 

Wednesday,  M24  9am-5pm 

Thursday,  M25-9am-5pm 

Summer!! 

Cooneratiye  Learning  - 

Multiculturalism  in  Course 

Grading 

9-11  Classroom  Research 

See  you  on: 

I^sentation 

Apply  MC  in  course 

What  Competencies  do  grades 

Experiments  - 

l.Summer- 

communicate? 

Research  date 

Manning  as  Assessment  & 

*Reyisit  Cooperatiye  Learning 

Plan  LC  Goals  and  Actiyities 

Reyiew  Teaching  & Learning 

Actiye  Learning  - Man  Courses 

Scholarship  of  Teaching 

2.Fall  dates: 

Complete  Assessment  System 

Comnlete  Assessment  System 

12-1  Lunch  with  Dean  Vohra 

a.  Regroup 

Comnlete  T/L  Decisions 

Program  Assessment  DISC. 

2:00-4:00  Dean  Vohra 

-Reyiew  TM/TS/CL/M 

Dept.  Chairs  Meeting/Profs. 

b.  MT/Final 

Mean/Use 

-Reyiew  St.  LS-Kolb  Model 
-Reyiew  Dale's  Cone- Analyze  A. 

What  does  your  Syllabus 
COMMUNICATE  - or  NOT!! 

Complete  Portfolio 
Teaching  & Learning  Assign. 

Program: 

Reflectiye  Practice  & Change 

Test  Analysis  & 
Reyiew  date 

Course  Content  Schedule 

The  MODEL  SYLLABUS 

Scholarship  of  Teach.  Assgn. 

Learning  Community 

c.  End  of 

Teaching  Decisions  - TM,  TS, 

Syllabus  Completion 

Teaching  & Learning 

Semester 

St.  LS 

Grading  Assignment-JS 

Discuss  Dean/Chair  Meeting 

Scholarship  of  Teaching 

d.  Data  Reyiew 

Course  Calendar  Completion 

*Portfolio  Assessment  (a) 

Set  Fall  LC  dates 

* Man  Program 

4-5pm  *Program  Assessment 

e.  Art.  Meet. 

Multicultural  Assignment  -JS 

*LC  Reassessment  - (b) 

*Self-Competency  Re-ass(b) 
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LITERATURE  THAT  INFORMED  THE  CEET  SCHOLARSHIP  OF  TEACHING 

INITIATIVE 

Jule  Dee  Scarborough,  Ph.D. 


Context 

Bruner  (1996)  discusses  the  culture  of  education  and  reminds  us  that  learning  is  not  just 
information  processing  nor  is  it  just  sorting  knowledge  into  categories.  The  purpose  of 
learning  is  to  construct  meanings  - not  to  merely  manage  information.  He  argues  for 
consideration  of  one’s  culture  and  the  use  of  narrative  as  an  “instrument  of  meaning 
making  - as  narrative  allows  for  the  understanding  of  the  past,  present,  and  future. 
Learning  within  culture  is  how  we  construct  our  world,  our  conception  of  ourselves,  and 
our  powers.  Human  mental  activity  is  not  one  that  occurs  solo,  nor  can  it  occur 
unassisted;  it  is  not  just  “inside  the  head”  (p.  xi).  Mental  life  requires,  and  is  lived,  with 
others.  Bruner  offers  tenets  and  consequences  for  education: 

1 . The  perspective  tenet.  First,  meaning  making.  The  meaning  of  any  fact, 
proposition,  or  encounter  is  relative  to  the  perspective  or  frame  of  reference  in 
terms  of  which  it  is  construed,  (p.  13) 

2.  The  contraints  tenet.  (a)The  forms  of  meaning  making  accessible  to  human 
beings  in  any  culture  are  constrained  in  two  crucial  ways.  One  first  inheres 
in  the  nature  of  human  mental  functioning  itself.  Our  evolution  as  a species 
has  specialized  us  into  certain  characteristic  ways  of  knowing,  thinking, 
feeling,  and  perceiving.  We  cannot,  even  given  our  most  imaginative 
efforts,  construct  a concept  of  Self  that  does  not  impute  some  causal  influence 
of  prior  mental  states  on  later  ones.  We  cannot  seem  to  accept  a version  of  our 
own  mental  lives  that  denies  what  we  thought  before  affects  what  we  think 
now.  We  are  obliged  to  experience  ourselves  as  invariant  across 
circumstances  and  continuous  across  time,  b)  In  the  second  those  constraints 
are  imposed  by  the  symbolic  systems  accessible  to  human  minds,  generally 
limits  imposed  by  the  very  nature  of  language  and,  more  particularly, 
constraints  imposed  by  the  different  languages  and  notational  systems 
accessible  to  different  cultures  (the  Whorf-Sapir  hypothesis)  that  thought  is 
shaped  by  the  language  in  which  it  is  formulated  and/or  expressed,  (pp.  15-18) 

3.  The  constructivism  tenet,  (implied  above)  The  “reality”  that  we  impute  to  the 
“worlds”  we  inhabit  is  a constructed  one  - “reality  is  made,  not 
found”(Goodman,  1992).  Reality  construction  is  the  product  of  meaning 
making  shaped  by  a culture’s  toolkit  of  ways  of  thought.  Education  must  be 
conceived  as  aiding  humans  in  learning  to  use  the  tools  of  meaning  making 
and  reality  construction  to  better  adapt  to  the  world  in  which  they  find 
themselves  and  to  the  process  of  changing  it  as  required.  ( Bruner,  1996,  p. 

19) 

4.  The  interactional  tenet.  Passing  on  knowledge  and  skill,  like  any  human 
exchange,  involves  a subcommunity  in  interaction,  (p.  20) 

5.  The  externalization  tenet.  Collective  cultural  activity  is  to  produce 
works...  [these  works]  help  make  a community  and  communities  of  mutual 
learners.  Works  and  works-in-progress  create  shared  and  negotiable  ways  of 
thinking  in  a group.  Externalizing  mental  work. . .produces  a “record”  of 
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mental  efforts,  one  that  is  “outside  us”  rather  than  vaguely  “in  memory”  [and] 

. . .relieves  us  from. . .“thinking  about  our  own  thoughts”  while  accomplishing 
the  same  end  - embodies  our  thoughts  and  intentions  in  a form  more 
accessible  to  reflective  efforts. . .the  process  of  thought  and  its  product 
become  interwoven,  (p.  22) 

6.  The  instrumentalism  tenet.  Education,  conducted  in  whatever  culture,  always 
has  consequences  in  the  later  lives  of  those  who  undergo  it. . . [and]  provides 
skills,  ways  of  thinking,  feeling,  and  speaking  that  later  may  be  traded  for 
“distinctions”  in  the  institutionalized  “markets”  of  a society.... Then, 
education  is  never  neutral,  never  without  social  and  economic 
consequences ...  .Education  is  always  political  in  the  broader  sense,  (p.  25) 

7.  The  institutional  tenet.  As  education  in  the  developed  world  becomes 
institutionalized,  it  behaves  as  institutions  do,  and  often  must,  and  suffers 
certain  problems  common  to  all  institutions.  Cultures  are  composed  of 
institutions  that  specify  more  concretely  what  roles  people  play  and  what 
status  and  respect  they  are  accorded.  Institutions  do  the  cultures’  serious 
business.  [They]  provide  the  “markets”  where  people  “trade”  their  acquired 
skills,  knowledge,  and  ways  of  constructing  meanings  for  “distinctions”  or 
privileges,  (pp.  29-30) 

8.  The  tenet  of  identify  and  self-esteem.  Perhaps  the  single  most  universal  thing 
about  human  experience  is  the  phenomenon  of  “Self,”  and  we  know  that 
education  is  crucial  to  its  formation.  Education  should  be  conducted  with  that 
fact  in  mind.  (p.  35) 

9.  The  narrative  tenet.  The  mode  of  thinking  and  feeling  that  helps. . .people 
create  a version  of  the  world  in  which,  psychologically,  they  can  envisage  a 
place  for  themselves  - a personal  world. . . .Story  making,  narrative,  is  what  is 
needed  for  that.  (p.  39) 

Bruner  (1996)  emphasizes  the  powers  of  consciousness,  reflection,  breadth  of  dialogue, 
and  negotiation  in  his  tenets.  They  pose  risks  because  they  seek  to  open  discussion  of  the 
current  institutional  authority.  He  mentions  that  the  tenets  and  education  are  both  risky  as 
they  “fuel  the  sense  of  possibility”  (p.  42).  He  feels  that  a failure  to  equip  minds  with  the 
skills  to  understand,  feel,  and  take  action  in  their  culture  is  not  just  pedagogically  zero, 
but  creates  alienation,  defiance,  and  incompetence  that  in  turn  undermine  a culture. 
Education  is  not  just  a technical  business  of  managing  information  processing  or  learning 
theories  and  then  assessing  their  achievement.  It  is  very  complex  where  members  of  a 
group  fit  to  achieve  the  needs  of  that  culture  (p.  43).  This  requires  authentic  teaching  and 
learning.  Palmer  (1993)  speaks  of  authenticity  as  the  creation  of  the  community  of  truth: 
“to  teach  is  to  create  a space  in  which  obedience  to  truth  is  practiced”  (p.  xvi).  Obedience 
is  removed  from  its  authoritarian  connotation.  This  requires  knowing  the  nature  of  reality 
where  community,  not  competition,  is  the  metaphor  from  science.  Palmer  mentions  that 
students  learn  as  much  from  the  “hidden  curriculum”  - institutional  patterns  and  practices 
- as  they  do  from  the  formal  curriculum.  He  advocates  that  the  image  of  reality  is  less 
individualistic  and  competitive  and  more  cooperative  and  communal  because  learning  is 
a communal  act  where  knowing  goes  beyond  relations  of  knowers  to  a community  of 
interaction  between  knowers  and  the  known,  where  there  is  a myth  of  objectivity,  where 
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the  “true  work  of  the  mind  is  to  reconnect  us  with  that  which  would  otherwise  be  out  of 
reach”  (p.  xvi).  He  says  that  some  students  resist  thinking  because  they  live  in  a world  of 
fragile  relationships;  that  these  students  are  desperate  for  more  community  and,  because 
of  that,  when  thinking  is  presented  to  them  as  a way  of  disconnecting  themselves  from 
each  other  and  the  world  (an  objective  approach),  they  do  not  want  to  engage.  He  feels 
they  want  to  “create  community,  a way  of  knowing  (engage  in  a community  of  learning)” 
(p.  xvii).  This  certainly  informs  our  initiative. 

The  seminal  work  by  Boyer  (1990)  led  the  Carnegie  Foundation  to  engage  in  the  Ernest 
L.  Boyer  Project  of  the  Carnegie  Foundation  for  the  Advancement  of  Teaching. ' Boyer 
raised  the  issue  of  how  faculty  spend  their  time  and  what  they  are  rewarded  for  doing. 
This  led  to  the  question:  "what  activities  of  the  professoriate  are  most  highly  prized. . . 
[noting]  that  it  is  futile  to  talk  about  improving  the  quality  of  teaching  if,  in  the  end, 
faculties  are  not  given  recognition  for  the  time  they  spend  with  students?"  (p.  xi).  He 
traces  the  debate  throughout  history,  illuminating  the  transitions  and  shifting  priorities  of 
American  higher  education,  noting  that  students  are  often  the  losers  and  further  noting 
that  students 

are  assured  that  teaching  is  important,  that  a spirit  of  community  pervades  the 
campus,  and  that  general  education  is  the  core  of  the  undergraduate 
experience. . . .but  the  reality  is  that,  on  far  too  many  campuses,  teaching  is  not 
well  rewarded,  and  faculty  who  spend  too  much  time  [working  with  students] 
may  diminish  their  prospects  for  tenure  and  promotion,  (p.  xii) 

Boyer’s  (1990)  goal  in  this  work  is  to  "break  out  of  the  tired  old  debate  and  define,  in 
more  creative  ways,  what  it  means  to  be  a scholar. . .recognize  the  full  range  of  faculty 
talent  and  the  great  diversity  of  functions  higher  education  must  perform...  [stating  that] 
for  American  higher  education  to  remain  vital,  we  urgently  need  a more  creative  view  of 
the  work  of  the  professoriate"  (p.  xii).  Most  important  in  his  introduction  is  his 
acknowledgement  of  the  "need  [for]  a climate  in  which  colleges  and  universities  are  less 
imitative,  taking  pride  in  their  uniqueness. . .to  end  the  suffocating  practice  in  which 
[they]  measure  themselves. . .by  external  status  rather  than  by  values  determined  by  their 
own  distinctive  mission"  (p.  xiii).  He  frames  the  question  of  better  education  in  the 
context  of  how  scholarship  is  defined  and  rewarded,  trying  to  reflect  what  he  and  others 
consider  the  full  range  of  academic  and  civic  mandates,  and  describes  four  views  of 
scholarship  - "discovery,  integration,  application,  and  teaching,"  defining  them  as  follows 
(pp.  xii-xiii). 

(1)  Knowledge  for  knowledge  sake  - the  creation  of  a bank  of  knowledge  or  information, 
ready  to  draw  upon  when  the  time  for  intelligence  use  arrives  (Thomas,  1977  as  cited  in 
Boyer,  1990). 

Scholarship  of  Discovery  comes  closest  to  what  academics  [identify]  as 
research. . .the  freedom  of  inquiry  and  to  follow,  in  a disciplined  fashion,  an 
investigation  wherever  it  may  lead. . . .Not  just  the  outcomes,  but  the  process,  and 
especially  the  passion,  give  meaning  to  the  effort. 


* This  information  is  also  replicated  in  the  introduction. 
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[Boyer  quotes]  Bowen  (1986),  “scholarly  research  reflects  our  pressing, 
irrepressible  need  as  human  beings  to  confront  the  unknown  and  to  seek 
understanding  for  its  own  sake. . .tied  inextricably  to  the  freedom  to  think  freshly, 
to  see  propositions  of  every  kind  in  the  ever  changing  light.  And  it  celebrates  the 
special  exhilaration  that  comes  from  a new  idea  (p.  17). . . . [T]he  probing  mind  of 
the  research  is  an  incalculably  vital  asset  to  the  academy  and  the  world. . .the  very 
heart  of  academic  life. . .the  pursuit  of  knowledge  must  be  assiduously  cultivated 
and  defended. . .the  discovery  of  new  knowledge  is  absolutely  critical,  (pp.  1 IT- 
US) 

(2)  Authenticating  knowledge  through  analysis  and  interpretation,  establishing  meaning 

or  original  research  through  interdisciplinary  consideration  and  synthesis. 

Scholarship  of  Integration  - the  need  for  scholars  who  give  meaning  to  isolated 
facts,  putting  them  in  perspective. . .making  connections  across  the  disciplines. . . 
serious,  disciplined  work  that  seeks  to  interpret,  draw  together,  and  bring  new 
insight  to  bear  on  original  research. 

[Boyer  quotes]  Van  Doren,  “[t]he  connectedness  of  things  is  what  the  educator 
contemplates  to  the  limit  of  his  [her]  capacity.”  It  is  through  connectedness  that 
research  ultimately  is  made  authentic. . .closely  related  to  discovery. . .where  fields 
converge. . . [where  one  fits]  one's  own  research  - or  the  research  of  others  - into 
larger  intellectual  patterns . . . .Those  engaged  in  discovery  ask  "What  is  to  be 
known,  what  is  yet  to  be  found?"  Those  engaged  in  integration  ask,  "What  do  the 
findings  mean!  and  provide  a . . .more  comprehensive  understanding. . .requiring 
critical  analysis  and  interpretation.”  (pp.  18-21) 

(3)  Where  scholarship  connects  theory  and  practice  and  proves  its  worth  to  the  nation 

and  world. 

Scholarship  of  Application  - How  can  the  knowledge  be  responsibly  applied  to 
consequential  problems?  How  can  it  be  helpful  to  individuals  as  well  as 
institutions?  Can  social  problems  themselves  define  an  agenda,  serving  the 
interests  of  the  larger  community. 

[When  considering  the  following  international  perspective  by  Harper  (1906)]  . . . 
Scholarship. . .was  regarded  by  the  British  as  "a  means  and  measure  of  self- 
development," by  the  Germans  as  "an  end  in  itself,  "but  by  Americans  as 
equipment  for  service."  Self-serving  though  it  [the  American  perspective]  may 
have  been,  this  analysis  had  more  than  a grain  of  truth. . .the  gap  between  the 
academy  and  the  needs  of  the  larger  world. . .service  is  routinely  praised,  but 
accorded  little  attention. 

Colleges  and  universities  have  rejected  service  as  serious  scholarship,  partly 
because  its  meaning  is  so  vague  and  often  disconnected  from  serious  intellectual 
work...[e.g.  projects,  committees,  etc.].  Clearly,  a distinction  must  be  drawn 
between  citizenship  activities  and  projects  that  relate  to  scholarship  itself. . . .To  be 
considered  scholarship,  service  activities  must  be  tied  directly  to  one's 
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professional  activity . . . serious,  demanding  work,  requiring  rigor. . . [and] 
accountability. . .associated  with  research  activities. . .The  process  we  have  in  mind 
is  more  dynamic  [where]  new  intellectual  understandings  arise  out  of  the  very  act 
of  application,  [where]  theory  and  practice  vitally  interact,  and  one  renews  the 
other...  both  apply[ing]  and  contribut[ing]  to  human  knowledge... [using]  the 
skills  and  insights  only  the  academy  can  provide. . . 

Handlin  observed  our  troubled  planet  "can  no  longer  afford  the  luxury  of  pursuits 
confined  to  an  ivory  tower”. . . [where]  scholarship  has  to  prove  its  worth  not  on  its 
own  terms  but  by  service  to  the  nation  and  the  world,  (as  cited  in  Boyer,  pp.  21-23 

Scholarship  of  Teaching  - [where]  the  work  of  the  professor  becomes 
consequential. ..  .as  it  is  understood  by  others . . .Today  teaching  is  often  viewed  as 
a function. . . [;  however,]  Aristotle  said,  "Teaching  is  the  highest  form  of 
understanding. "....beginning  with  what  the  teacher  knows. . .steeped  in  the 
knowledge  of  their  fields . . . .One  reason  why  legislators  fail  to  understand  why  10- 
12  hours  in  the  classroom  each  week  can  be  a heavy  load  is  their  lack  of 
awareness  of  the  hard  work  and  serious  study  that  undergirds  good  teaching,. . .a 
dynamic  endeavor  involving  all  analogies,  metaphors,  and  images  that  build 
bridges  between  the  teacher's  understanding  and  the  student's  learning. 
Pedagogical  [and  adrogogical]  procedures  must  be  carefully  planned  and 
continuously  examined. . . [According  to]  Palmer  (1983). . .knowing  and  learning 
are  communal  acts.  With  this  vision,  great  teachers  create  a common  ground  of 
intellectual  commitment.  They  stimulate  active,  not  passive,  learning  and 
encourage  students  to  be  critical,  creative  thinkers,  with  the  capacity  to  go  on 
learning  after  their  college  days  are  over.  Further,  good  teaching  means  that 
faculty,  as  scholars,  are  also  learners.... [not]  transmit[ting]  information  that 
students  are  expected  to  memorize  and  then  recall. . .but  transforming  and 
extending  it  as  well. . .Inspired  teaching  keeps  the  flame  of  scholarship  alive. . .All 
academics  credit  good  teachers. . .defining  their  work  so  compellingly  that  it 
became. . .a  lifetime  challenge.  Without  the  teaching  function,  the  continuity  of 
knowledge  will  be  broken  and  the  store  of  human  knowledge  dangerously 
diminished. 

Oppenheimer  (1954)  noted... "The  specialization  of  science  is  an  inevitable 
accompaniment  of  progress;  yet  it  is  full  of  dangers,  and  it  is  cruelly  waster,  since 
so  much  that  is  beautiful  and  enlightening  is  cut  off  from  the  rest  of  the  world. 
Thus,  it  is  proper  to  the  role  of  the  scientist  that  he  [she]  not  merely  find  the  truth 
and  communicate  it  to  his  [her]  fellows,  but  that  he[she]  teach,  that  he  [she]  try  to 
bring  the  most  honest  and  most  intelligible  account  of  new  knowledge  to  all  who 
will  try  to  learn. . .knowledge  is  acquired  through  research,  synthesis,  practice,  and 
through  teaching,  (as  cited  in  Boyer,  1990,  pp.  23-24) 

These  four  types  of  scholarship  acknowledge  the  great  range  of  talent  and  diversity 
within  the  professoriate.  The  creative  tension  between  the  above  definitions  stimulates  us 
to  appreciate  scholarship  from  a broader  perspective,  each  type  contributing  significantly 
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to  the  other  and  ultimately  to  the  development  of  humanity  and  its  endeavors  through  the 
academy  and  other  contributing  institutions. 

Also  important  to  us  is  Boyer's  (1990)  point  that  there  is  a unique  opportunity  for 
comprehensive  universities,  one  where  we  can  establish  our  own  unique  missions  rather 
than  imitate  the  traditional  research  universities,  the  opportunity  to  blend  quality  with 
innovation,  choosing  the  foci  of  our  passion  (whether  Scholarship  of  Discovery, 
Scholarship  of  Integration,  Scholarship  of  Application,  or  the  Scholarship  of  Teaching) 
with  the  understanding  that  the  Scholarship  of  Teaching  is  a requirement  for  knowledge 
to  continue  to  expand  and  be  used.  We  agree  with  Boyer  that  "diversity  with  dignity"  (p. 
64)  is  building  a diverse  learning  system  and  learning  organization  where  undergraduate 
and  graduate  learning  are  priorities;  where  the  Scholarship  of  Teaching  is  honored  and 
prioritized  in  conjunction  with  other  types  of  scholarship;  where  they  integrate,  one  not 
dominating  the  other;  and  where  the  diverse  range  is  sought,  sustained,  and  respected. 
Thus,  not  only  is  it  mandatory  that  we  move  ahead  to  formally  acknowledge  the 
importance  of  the  Scholarship  of  Teaching  alongside  the  others,  it  also  reaffirms  and 
acknowledges  the  importance  of  the  historical  commitment  to  teaching  and  student 
learning  in  our  service  to  secondary  education  and  extends  that  work  into  our  own 
engineering  and  technology  classrooms  at  the  university  level  in  an  informed  manner, 
with  an  understanding  of  what  our  vision  means,  what  will  be  required  to  attain  it,  and 
that  the  extension  of  knowledge  rewards  what  will  ensue.  This  vision  attends  to  preparing 
students  for  their  professions  as  learning  individuals,  integrating  general  and  major 
education  more  relevantly  and  overtly,  strengthening  capstone  experiences,  and 
ultimately  examining  the  master's  degree  educational  experiences  to  determine  what 
needs  to  be  sustained  or  strengthened.  Most  important  is  to  establish  a more  productive 
relationship  between  teaching  and  student  learning.  Therefore,  we  engage  in  this 
initiative  to  actively  explore  that  relationship  through  teaching  and  learning  research.  We 
will  begin  simply  with  a two-pronged,  but  interdependent,  research  initiative:  the  first 
research  focus  will  be  the  design,  development,  and  piloting  of  a new  faculty 
development  model  and  program  to  prepare  a pilot  group  of  faculty  to  actively  engage  in 
experimental  research  on  classroom  teaching  and  student  learning;  the  second  research 
focus  will  be  the  actual  experimental  classroom  research  on  teaching  and  student  learning 
(TL).  Both  of  these  activities  will  be  rigorously  evaluated  to  determine  their  value  to 
inform  the  faculty  and  administration  about  what  the  next  level  of  activities  should  be  to 
sustain,  expand,  and  deepen  the  Scholarship  of  Teaching  initiative.  As  Boyer  (1990) 
establishes,  we  have  the  opportunity  to  determine  our  own  unique  model  and  what  is 
acceptable  as  faculty  role,  function,  and  responsibility. 

Glassick,  Huber,  and  Maeroff  (1997)  expanded  Boyer's  (1990)  conceptual  standards  for 
faculty  research  by  engaging  in  a project  to  assess  scholarship  by  the  professoriate.  One 
of  the  top  priorities  of  the  Carnegie  Foundation  was  to  strengthen  undergraduate 
education.  Boyer  and  his  colleagues  tried  to  clarify  the  purposes  of  higher  education, 
explore  what  constitutes  quality,  and  examine  its  functions.  Teaching  became  less  well 
rewarded  after  World  War  II,  and  service  changed  from  a tradition  of  extending 
knowledge  beyond  the  campus  to  a variety  of  less  important  activities.  Sharing 
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knowledge  through  teaching  is  no  longer  prestigious  and  research  has  become  the 
priority. 

Clark  (1987)  questioned  why  research  became  the  only  form  of  scholarship  considered 
worthy  when  “the  greatest  paradox  of  academic  work  in  modern  America  is  that  most 
professors  teach  most  of  the  time,  and  large  proportions  of  them  teach  all  the  time,  but 
teaching  is  not  the  activity  most  rewarded  by  the  academic  profession  nor  most  valued  by 
the  system  at  large”  (pp.  98-99).  Those  at  the  Carnegie  Foundation  submit  that  the 
skewing  of  the  mission  of  higher  education  toward  research  has  created  a kind  “crises  of 
purpose”  in  American  universities,  leading  to  other  functions  (e.g.,  teaching)  becoming 
undervalued  and  resulting  in  high  costs  to  undergraduate  education.  Boyer  and  Levine 
(1981)  argued  that  the  gap  between  rhetoric  and  the  reality  of  curriculum  must  be  closed 
and  that  connections  to  and  understanding  the  world  must  again  become  a priority. 
Another  aspect  of  the  academy  has  suffered  - that  of  the  application  of  knowledge 
through  professional  service  from  too  narrow  a definition  of  research.  Boyer's  (1990) 
goal  was  to  move  beyond  this  research  versus  teaching  debate  and  provide  a more 
efficacious  meaning  for  research  and  to  identify  the  work  of  faculty  in  ways  that  enrich 
the  quality  of  undergraduate  education,  where  there  is  “an  educationally  purposeful 
community,  a place  where  faculty  and  students  share  academic  goals  and  work  together 
to  strengthen  teaching  and  learning”  (p.  12).  Glassick  et  al.  (1997)  continue  Boyer's  line 
of  thought  but  move  beyond  the  conceptual  to  the  development  of  criteria  and  procedures 
for  each  type  of  research  through  extensive  research  of  the  literature,  handbooks,  policy 
statements,  faculty  consultations,  conferences,  survey  research,  and  more  while  collecting 
data  from  all  four-year  colleges  and  universities  and  600  on  faculty  evaluation.  Glassick 
et  al.  establish  that  all  four  types  of  research  must  be  held  to  the  same  standards  of 
scholarly  quality  performance  rather  than  accepting  the  current  practice  that  different 
types  of  standards  apply  to  different  types  of  faculty  work.  There  is  an  emerging  climate 
to  support  the  idea  of  the  different  types  of  scholarly  work.  Faculty  members  are 
accumulating  evidence  of  achievements  across  the  four  types  of  research.  What  then  are 
the  common  dimensions  of  scholarly  work?  The  authors'  research  clearly  showed  that 
when  a work  of  scholarship  is  appreciated,  faculty  agree  that  it  has  been  guided  by  clear 
goals  and  adequate  preparation,  has  engaged  appropriate  methodology,  has  accomplished 
significant  results,  was  presented  effectively,  and  involved  reflective  critique.  Glassick  et 
al.  go  further  to  describe  each  of  these  qualitative  standards: 

• Clear  Goals  - Does  the  scholar  state  the  basic  purposes  of  his  or  her  work  clearly? 
Does  the  scholar  define  objectives  that  are  realistic  and  achievable?  Does  the 
scholar  identify  important  questions  in  the  field? 

• Adequate  Preparation  - Does  the  scholar  show  an  understanding  of  the  existing 
scholarship  in  the  field?  Does  the  scholar  bring  the  necessary  skills  to  his  or  her 
work?  Does  the  scholar  bring  together  the  resources  necessary  to  move  the  project 
forward? 

• Appropriate  Methods  - Does  the  scholar  use  methods  appropriate  to  the  goals? 
Does  the  scholar  effectively  apply  the  methods  selected?  Does  the  scholar  modify 
procedures  in  response  to  changing  circumstances? 
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• Significant  Results  - Does  the  scholar  achieve  the  goals?  Does  the  scholar's  work 
add  consequentially  to  the  field?  Does  the  scholar's  work  open  additional  areas  for 
further  exploration? 

• Effective  Presentation  - Does  the  scholar  use  a suitable  style  and  effective 
organization  to  present  his  or  her  work?  Does  the  scholar  use  appropriate  forums 
for  communicating  work  to  its  intended  audiences?  Does  the  scholar  present 
his/her  message  with  clarity  and  integrity? 

• Reflective  Critique  - Does  the  scholar  critically  evaluate  his  or  her  own  work? 
Does  the  scholar  bring  an  appropriate  breadth  of  evidence  to  his  or  her  critique? 
Does  the  scholar  use  evaluation  to  improve  the  quality  of  future  work?  (pp.  22- 
36) 

This,  then,  becomes  the  common  language  with  which  to  discuss  standards  for  scholarly 
work  - all  types  of  scholarly  work.  This  language  helps  establish  what  the  four  types  of 
research  (discovery,  integration,  application,  and  teaching)  share  as  scholarly  activities. 

Providing  evidence  of  quality  or  documenting  scholarship  so  the  evidence  directly  shows 
the  standards  have  been  met  requires  a wide  variety  of  materials.  Some  thoughtful 
educators  feel  that  well  prepared  documentation  promotes  better  scholarship  because  it 
engages  them  in  the  justification  of  their  achievements.  This,  in  turn,  promotes  more 
reflective  practice  that  leads  to  improvements.  Once  again,  this  must  equally  apply  to  all 
types  of  research.  A challenge  may  be  that  an  expanded  definition  of  research  leads  to 
concerns  about  the  types  and  sources  of  documentation  necessary  to  provide  evidence  of 
quality  research.  However,  for  example,  discovery  or  traditional  research  is  more  easily 
documented  with  peer  review  processes,  publication  criteria,  etc.  already  in  place. 
Discovery  research  results  are  more  easily  documented.  The  process  of  teaching  may  not 
be  as  easily  documented  (e.g.,  working  with  students  in  an  office,  preparation  for 
teaching,  the  follow-through  process,  elements  not  in  print  or  easily  objectively 
documented).  Social  science  research  (research  involving  human  interaction,  institutional 
constraints,  dynamic  organization,  etc.)  is  not  as  easily  implemented  as  a laboratory 
experiment  in  the  sciences,  engineering,  or  technology.  In  fact,  one  of  the  greatest 
difficulties  educational  researchers  have  is  where  colleagues  not  engaged  in  educational 
research  do  not  understand  the  context  of  the  research,  and  even  more  impacting  is  that 
educational  research  may  require  more  time  and  different  designs  and  methodology. 
Furthermore  those  from  the  sciences  and  engineering  remain  suspicious  when  confronted 
with  the  more  qualitative  aspects  of  the  research  and  methodologies.  Therefore, 
acceptable  documentation  must  include  a more  "eclectic"  array  of  sources  and  types.  This 
is  the  only  way  the  newer  forms  of  scholarship  can  be  treated  fairly  (e.g.,  integration, 
application,  and  teaching).  Regarding  teaching,  there  has  long  been  agreement  that 
documentation  can  be  more  creative  - for  example,  a variety  of  portfolios.  Portfolios  are 
a good  example  of  appropriate  documentation  on  the  scholarship  of  teaching  and,  when 
designed  as  assessment  and  evaluation  documentation  and  include  a process  of  integrity, 
can  easily  reveal  the  results  of  implemented  changes  - what  has  been  learned.  Therefore, 
there  needs  to  be  in-depth  consideration  of  what  types  of  information  or  research 
evidence  will  provide  clear  documentation  of  the  research  results.  In  addition,  there  must 
be  a climate  of  trust  between  the  faculty  and  administration,  with  clear  understanding 
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about  the  expectations  of  performance.  This  will  go  far  to  continue  to  recognize  that 
professors  “have  special  responsibilities  to  their  disciplines,  their  students,  their 
colleagues,  and  their  institutions”:  integrity,  professional  ethics,  perseverance,  and 
courage  (Glassick  et  ah,  1997,  p.  61). 

There  is  a dearth  of  authors  who,  throughout  the  nineties  and  now,  continue  to  discuss  the 
roles  of  higher  education  faculty,  their  academic  work,  and  the  rewards  for  their  work. 
The  stimulus  for  much  of  that  discussion  is  about  faculty  teaching,  whether  teaching  in 
higher  education  is  a real  priority  and  how  the  reward  structure  seems  to  reflect  it  as  a 
lesser  priority  when  a faculty  member  is  considering  promotion,  tenure,  prestige,  and 
"power"  within  the  academy  (Gallos  as  cited  in  Frost  & Taylor,  1996).  Whicker  et  al. 
(1993  as  cited  in  Frost  & Taylor)  describe  the  "publish-or-perish"  situation  at  major 
universities,  where  less  than  adequate  teaching  may  cause  difficulty  in  being  awarded 
tenure;  however,  an  outstanding  publishing,  research,  or  grants  record  may  compensate 
for  weak  or  mediocre  teaching.  Conversely,  excellent  teaching  is  not  often  sufficient  to 
compensate  for  inadequate  publishing,  research,  or  grant  awards.  Although  most 
universities  have  the  criterion  of  adequate  teaching  along  side  research  and  scholarship 
and  service,  the  latter  is  much  more  significantly  weighted.  Meeting  the  research  criterion 
also  involves  getting  grants  that  are  expected  to  produce  results  that  end  up  published  by 
acceptable  sources.  Greater  grant  awardees  are  more  tenurable  than  those  who  have 
fewer,  and  worse  yet,  faculty  members  may  be  awarded  tenure  or  promotion  based  upon 
a high  record  of  grant  awards  with  no  publication  record  to  validate  results  or  outcomes. 
Having  brought  that  issue  to  the  discussion,  it  might  be  important  to  give  caution.  Grants 
are  awarded  by  agencies  or  organizations  that  may  prohibit  publication;  or  grant 
participants  (e.g.,  schools)  may  not  be  willing  to  engage  if  the  results  are  to  be  published. 
Even  though  public,  (e.g.,  state  or  federal  grants)  are  awarded  openly  and  have  no 
restrictions  about  publication,  there  are  times  when  grants  produce  important  results  that 
cannot  be  published.  The  reward  system  needs  to  work  for  faculty  members  who  may  be 
caught  in  those  situations  or  those  who  are  doing  highly  sensitive  research  with  private 
parties  for  which  proprietary  information  cannot  be  published  by  the  professor(s). 
Colleges  and  departments  need  to  carefully  address  these  situations  and  figure  out  how  to 
accommodate  "grants"  or  "contracts"  for  which  it  may  not  be  possible  to  validate  their 
results  and  outcomes  through  publication.  Braxton  (1996)  leads  a rich  discussion  through 
nine  chapters  by  various  authors  on  the  relationship  between  teaching  and  research.  In 
this  volume,  Braxton  discusses  the  conflict  between  research  and  teaching  and  brings 
forth  Boyer's  (1990)  comments  that  to  accommodate  research  activities  by  professors, 
their  teaching  loads  are  reduced,  teaching  assistants  are  assigned  to  large  undergraduate 
courses,  and  thus  undergraduate  needs  are  ignored.  Braxton  brings  to  bear  the  public 
expectation  that  faculty  members  spend  more  of  their  time  and  efforts  on  teaching  than 
scholarship  activities  and  discusses  the  conflict  or  competition  between  the  two  faculty 
functions  from  contrasting  perspectives:  null,  conflict,  and  complementarity  (p.  6). 

• Null  - This  perspective  posits  that  there  is  no  relationship  between  teaching  and 
research;  the  roles  are  independent  of  one  another  (Finkelstein,  1984)  and  do  not 
detract  from  one  another  (Linsky  & Straus,  1975).  Thus,  the  null  relationship  may 
be  the  inverse  of  complementarity.  More  specifically,  teaching  and  research  may 
be  clearly  set  apart  from  general  ability,  professional  goals,  and  values.  Also 
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there  may  be  a lack  of  correspondence  between  teaching  and  research 
specializations.  Moreover,  these  two  roles  may  not  be  mutually  reinforcing. 

• Conflict  - There  are  several  variations  of  the  position  that  a negative  relationship 
exists  between  teaching  and  research.  One  is  that  the  roles  of  teaching  and 
research  conflict  because  they  carry  different  expectations  and  different 
obligations  (Fox,  1992).  Another  variation  is  the  assertion  that  the  allocation  of 
time  sums  to  zero.  Because  productive  scholars  spend  more  time  in  research,  they 
spend  less  time  in  teaching.  Consequently,  the  quality  of  teaching  is  adversely 
affected  by  the  lesser  amount  of  time  spent  in  teaching  activities  (Finkelstein, 
1984).  A third  variant  is  that  teaching  and  research  require  different  abilities  and 
personality  traits.  (Finkelstsein,  1984;  Linsky  & Straus,  1975) 

• Complementarity  - refers  to  the  extent  to  which  the  roles  of  teaching  and  research 
are  similar  on  several  dimensions  (Faia,  1976).  General  ability  is  one  such 
dimension.  Teaching  and  research  require  the  same  general  ability  in  the 
enactment  of  these  two  roles  (Finkelstein,  1984;  Linsky  & Straus,  1975).  Because 
the  academic  profession’s  basic  goal  of  "furthering  knowledge"  can  be  realized 
through  both  research  and  teaching,  professional  goals  represent  another 
dimension  on  which  similarity  may  exist  (Faia,  1976).  Values  are  another  point  of 
possible  similarity  (Fox,  1992;  Parsons  & Platt,  1973;  Shils,  1983).  Holding 
cognitive  rationality  as  a value,  for  example,  suggests  favoring  an  integration  of 
the  roles  of  teaching  and  research  (Parsons  & Platt,  1968).  Cognitive  rationality  is 
the  comprehension  and  solution  of  intellectual  problems  in  rational  terms  (Platt, 
Parsons,  & Kirshtein,  1976);  this  value  pattern  manifests  itself  in  both  teaching 
and  research  (Braxton,  1983). 

Although  not  a lot,  there  is  some  research  to  support  the  relationship  between  teaching 
and  research.  Feldman's  (1987)  meta-analysis  is  the  most  comprehensive  and  most  recent. 
He  found,  after  reviewing  29  studies,  that  there  was  an  average  correlation  of  -I-.12 
(p<.001)  between  research  productivity  and  student  assessments  of  teaching 
effectiveness,  thus  some  support  for  complementarity  between  research  and  teaching. 
Braxton  (1997)  added  the  Voeks  (1962)  study  and  also  addressed  the  question  of  the 
extent  of  support  for  the  three  types  of  perspectives.  He  found  that  both  the 
complementarity  (11)  and  null  (18)  perspectives  receive  moderate  support.  However,  the 
conflict  perspective  (1)  received  weak  support.  When  Feldman  executed  his  analysis,  he 
classified  each  institution  by  Carnegie  classification;  the  null  perspective  receives  strong 
affirmation  in  research  universities.  The  null  and  complementarity  perspectives  have 
modest  support  in  the  remaining  four  classifications:  doctoral,  comprehensive,  liberal 
arts,  and  unspecified.  He  concluded  that  “research  does  not  interfere  with  teaching 
effectiveness,”  and  “a  systematic  relationship  between  teaching  and  research  role 
performance  does  not  exist  across  different  types  of  colleges  and  universities”  (p.  8).  This 
raises  yet  more  questions  about  the  state  of  affairs  between  these  two  roles  or  functions. 

Is  the  null  situation  a natural  condition?  If  complementarity  exists,  do  individual  faculty 
characteristics  or  culture  contribute  to  it?  Do  faculty’s  professional  goals  and  values 
make  the  difference?  If  faculty  members  have  a dual  orientation  toward  both  roles. 
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complementarity  may  then  exist,  or  could  differences  occur  across  organizational 
cultures,  especially  if  the  two  roles  are  equally  valued.  Sullivan  goes  further  to  consider 
teaching  norms  and  publication  productivity,  finding  that  there  is  no  significant 
difference  in  teaching  support  and  publication  productivity  and  “the  mechanisms  of 
social  control  for  teaching  are  not  attenuated  by  high  levels  of  publication 
productivity.... Apparently  faculty  members  recognize  and  acknowledge  norms  that 
support  teaching,  while  simultaneously  participating  in  the  research  activities  expected  in 
today's  academic  environment”  (as  cited  in  Braxton,  1996,  p.  19).  This  indicates  a more 
positive  view  to  an  ongoing  discussion  that  often  seems  to  have  a negative  tone. 

The  authors  of  this  volume  go  on  to  discuss  teaching  norms  and  publication  productivity; 
research  activity  and  the  support  of  undergraduate  education;  the  research  versus  teaching 
debate;  untangling  the  relationships;  faculty  productivity  and  the  complexity  of  student 
exam  questions;  triangulating  the  relationships  among  publication  productivity,  teaching 
effectiveness  and  student  achievement;  institutional  and  departmental  cultures:  the 
relationship  between  teaching  and  research;  and  framing  the  public  policy  debate  on 
faculty:  what  is  the  role  of  research?  This  volume  ends  with  a piece  on  public  trust, 
research  activity,  and  the  idea  of  service  to  students  as  clients  of  teaching.  Olsen  and 
Simmons  (as  cited  in  Braxton,  1996)  mention  Feldman's  (1987)  results  as  disappointing 
and  note  that  public  concern  over  the  rising  costs  and  quality  of  undergraduate  education 
was  rising,  especially  when  considering  the  impact  of  research  and  its  supporting  reward 
system  on  instruction  (Fairweather,  1993;  Winston,  1994).  Goode  (1960)  and  Merton 
(1957)  introduce  "role  conflict"  as  a theory  in  which  proportionately  less  time  and  energy 
is  spent  on  one  role  as  more  is  invested  in  another.  Thus,  faculty  members  who  teach 
where  research  is  more  valued  and  rewarded  spend  less  time  and  energy  on  teaching. 

Diamond  and  Adam  (1993),  Fairweather  (1993),  Fairweather  and  Rhodes  (1995),  and 
Moore  and  Amey  (1993)  agree  that  faculty  reward  systems,  salary  assignments,  and 
prestige  constraints  undermine  professional  investment  in  teaching.  Massey  and  Wilger 
(1995)  go  further  to  point  out  that  even  where  the  quality  of  teaching  is  relatively  high, 
more  time  and  effort  would  still  improve  the  quality  of  undergraduate  education.  Other 
issues  abound.  Faculty  interested  in  research  may  be  less  interested  in  undergraduate 
curriculum  if  perceived  as  less  intellectually  stimulating  and  thus  not  as  tuned  in  to 
students'  needs.  Finally,  research  often  requires  isolated  and  focused  time  and  solitude, 
which  does  not  lend  itself  to  making  time  for  students  (Faia,  1976;  Fairweather,  1993; 
Feldman,  1987;  Friedrich  & Michalak,  1983).  Conversely,  some  theories  endorse  the 
complementarity  perspective  as  one  in  which  there  are  synergies  between  the  research 
and  teaching  roles  or  functions  and  that  those  roles  are  mutually  supportive.  For  example, 
active  researchers  more  often  keep  their  courses  infused  with  current  developments  or 
information,  hold  their  students  to  higher  standards,  communicate  their  enthusiasm,  and 
seem  to  have  a better  sense  of  knowledge  growing  over  time  (Centra,  1983;  Sieber,  1974; 
Thoits,  1987).  They,  again,  mention  the  Feldman  study  (1987),  as  he  identified  four 
instructional  dimensions  to  be  most  strongly  associated  with  research  productivity: 
knowledge  of  subject  matter;  intellectual  expansiveness;  preparation  and  organization  of 
course;  and,  clarity  of  course  objectives  and  requirements.  However,  they  also  mention 
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that  Feldman's  study  indicated  that  not  all  aspects  of  teaching  and  research  are  closely 
aligned  (e.g.,  student  rapport  and  research  productivity). 

Volkwein  and  Carbone  (1994)  studied  a different  issue  of  department  culture  and  reward 
system  orientation.  They  found  that  when  departments  valued  both  research  and  teaching, 
there  were  positive  academic  outcomes  for  students,  but  when  considering  the 
relationship  between  research  productivity  and  instructional  methodology  by  faculty 
members,  the  research  is  vague.  Once  again,  Feldman  (1987)  posited  that  pedagogical 
skills  may  be  enhanced  by  research  productivity  (e.g.,  knowledge,  organization, 
intellectual  expansiveness,  and  clarity).  Olson  and  Simmons's  (as  cited  in  Braxton,  1996) 
study  measured  research  productivity  over  a multi-year  timeframe,  employing 
institutional  data  used  to  make  salary  and  other  decisions.  The  study  also  considers  actual 
data  about  actual  classroom  practices,  which  goes  beyond  the  other  studies  assessing 
teaching  effectiveness  using  summative  student  evaluations.  Finally,  also  different,  Olson 
and  Simmons  executed  their  study  in  a large  research  institution.  They  found  that  faculty 
spent  44%  of  their  time  on  teaching  and  34%  on  research,  figures  close  to  those  reported 
nationally  by  Research  I institutions  at  which  43%  is  spent  on  teaching  and  29%  on 
teaching  (National  Center  for  Educational  Statistics,  1990).  These  authors  present  that 
their  data  indicated  a significant  negative  relationship  between  the  amount  of  time  spent 
on  research  and  teaching  and  conclude  that  their  results  failed  to  support  the  current  view 
that  many  positive  teaching  behaviors  are  inversely  related  to  research  productivity,  that 
faculty  research  faculty  did  not  avoid  teaching  lower- level  undergraduate  classes,  that 
faculty  did  not  rely  more  on  lecture  and  less  on  active  learning  techniques,  and  that 
research  faculty  did  not  use  more  multiple-choice  tests  in  their  courses  than  other  faculty 
members.  However,  Olson  and  Simmons  did  not  provide  significant  evidence  that  more 
productive  researchers  demonstrate  higher  levels  of  pedagogical  skills  and  that  faculty 
with  higher  research  productivity  did  seem  to  spend  less  contact  time  with  students.  They 
sum  it  up  by  suggesting  that  research  and  teaching  performance  are  unrelated  when 
teaching  is  defined  by  instructional  practices  and  are  negatively  related  when  some 
aspects  of  student-faculty  contact  are  considered.  They  recommend  that  contact  time 
between  faculty  and  students  benefits  both,  that  active  learning  techniques  should  be 
encouraged,  that  the  teaching  evaluation  process  needs  to  be  broadened,  and  that 
undergraduates  should  be  provided  the  opportunity  to  learn  first  hand  about  faculty 
research,  linking  teaching  and  research.  Also  in  Braxton  (1996),  Johnson’s  results 
considering  faculty  productivity  and  the  complexity  of  student  exam  questions  revealed 
that  “scholars  who  publish  books  and  those  who  publish  fewer  articles  ask  more  critical- 
thinking  questions  and  that  publishing  more  articles  seemed  to  negatively  affect  the 
asking  of  such  questions”  on  student  exams  (p.  41). 

Andre  and  Frost  (1996)  offer  many  interesting  and  thoughtful  discussions  throughout  the 
individual  chapters  from  perspectives  of  "researchers  hooked  on  teaching."  For  example, 
Fukami  begins  by  seeing  both  research  and  teaching  as  requiring  curiosity  and  humility 
(the  ability  to  recognize  that  you  do  not  know  everything)  and  confidence  in  what  you 
think,  as  well  as  the  ability  to  break  complexities  into  more  manageable  pieces  and  to 
identify  new  issues  and  intrinsic  motivation.  Teaching  and  research  "are  merely  two 
actions  we  take  in  playing  out  one  role:  a faculty  member  who  creates  and  disseminates 
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knowledge"  (as  cited  in  Andre  & Frost,  1996,  p.  5).  Also  important,  Fukami  notes 
teaching  and  research  can  represent  two  different  roles  and  real  differences: 

1 . “Teaching  is  immediate  and  research  long-run,”  where  teaching  immediately 
reveals  whether  students  are  learning  (if  there  is  adequate  and  timely  student 
assessment  in  place)  and  research  is  concluded  and  presented  to  the  community  at 
large  over  a great  deal  of  time,  often  years  between  the  beginning  of  a research 
project  and  results  dissemination. 

2.  “Teaching  is  forgiving,  whereas  research  is  cruel,”  where  mistakes  in  teaching 
can  be  rectified,  but  the  research  and  publishing  or  disseminating  process  is 
difficult,  often  a highly  anxious  and  long  or  short  window  process. 

3.  I feel  confident  that  my  teaching  makes  a very  real  impact,  whereas  research  (at 
least  in  the  way  we  currently  perform  it)  may  never  affect  anything  real.  (pp.  4-5) 

Fukami  concludes  with  lessons  learned,  mentioning  that  balance  is  important  but  may  not 
be  calm,  that  one  may  be  committed  to  more  than  one  object  or  identify  with  one 
organizational  membership  (as  cited  in  Andre  & Frost,  1996).  She  (with  others) 
conducted  a study  on  unionized  workers  and  whether  they  could  be  loyal  to  both  the 
company  and  union  and  found  that,  in  fact,  performance  was  higher  and  absenteeism 
lower  for  workers  committed  to  both  company  and  union  over  workers  committed  to  only 
one.  The  results  were  similar  with  nurses  when  considering  commitment  to  the 
profession,  hospital,  or  coworkers.  These  studies  may  have  implications  for  higher 
education:  faculty  members  can  have  multiple  and  concurrent  commitments.  Certainly, 
we  have  at  least  one  example  of  that. 

Gutek,  another  author  in  Andre  and  Frost  (1996),  believes  that  teaching  and  research  are 
inexorably  linked. . .view[ing]  it  as  primarily  a way  to  explain  research  to 
others . . . [that]  research  findings  would  never  be  disseminated  to  the  public 
through  journals,  nor  were  they  ever  intended  to  do  that.  They  are  too  dull,  boring, 
tedious,  and  difficult  to  understand  for  anyone  not  trained  in  research. . . .Teaching 
[presents]  real  possibilities  as  a way  to  inform  young  minds  about  the  progress 
being  made  on  the  research  front. . ..[However],  my  views  about  the  role  and 
purpose  for  teaching  have  changed. . ..I  now  worry  that  the  wonderful  interplay  of 
teaching  and  research  is  being  threatened  by  changes  in  higher  education,  (p.  27) 

It  is  important  to  understand  how  Gutek  began  teaching,  as  it  is  the  opinion  of  this  author 
(Scarborough)  that  it  is  one  of  the  very  best  methods  and  one  that  she  has  used 
throughout  the  teaching  aspect  of  her  career.  Gutek  (and  Scarborough)  immerse 
themselves  in  the  literature  to  identify  course  content,  as  they  both  feel  there  is  never  a 
textbook  or  "package"  that  agrees  with  their  idea  of  the  content  or  materials  needed. 
(Scarborough  has  included  texts  as  requirements;  she  does  not  use  them  as  the  content  but 
rather  as  one  validation  source  of  content  chosen.)  So  the  review  and  summarization  of 
the  literature  is  one  of  their  methods  of  identifying  the  research,  theories,  and  information 
they  wish  to  cover. 

At  one  point  Gutek  did  not  believe  in  case  studies  as  a student  research  activity  and  a 
method  to  validate  the  conceptual  or  theoretical  content;  however,  in  moving  from  a 
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psychology  department  to  a business  management  department,  Gutek  realized  that 
concepts  and  theories  are  not  enough  to  explain  theory  and  concepts.  One  might  describe 
it  in  this  way:  the  conceptual  and  theoretical  are  one  thread  and  the  real  world  cases  are 
another,  but  both  threads  are  required  to  weave  the  whole  fabric  of  understanding.  This  is 
in  line  with  Scarborough's  long  practice  of  weaving  the  theory  with  practice  and  practice 
in  the  field  or  "communities  of  practice."  Cases  present  opportunities  to  analyze  what  is 
going  on  and  actually  provide  a context  within  which  to  analyze  theories,  concepts, 
principles,  or  information  taught.  Gutek  feels  that  professors  need  to  cover  content  they 
really  know  themselves,  not  just  that  from  others’  textbooks,  and  to  present  as  many 
perspectives  as  possible.  We  both  agree  that  it  is  important  to  critically  analyze  content, 
regardless  of  the  position  on  content. 

Gutek,  however,  has  undergone  a theoretical  change  about  teaching  - professor  as  expert 
(content  more  important  than  process).  That  model  is  heavily  criticized  today,  especially 
since  students  come  to  us  with  a different  perspective.  In  Gutek’ s case,  and  mine  as  well, 
most  of  our  students  are  working  part  to  full  time  while  engaged  in  courses  or  degree 
programs  for  a variety  of  reasons.  A change  in  student  focus  makes  us  reconsider  what 
we  teach  and  how  we  teach  it  as  well.  She  mentions  that  when  moving  from  psychology 
to  the  business  college,  her  students  were  more  focused  on  getting  a job  and  what  the 
employers  expected  them  to  learn  - education  is  an  investment  in  their  future  and  that 
knowledge  of  research  may  or  may  not  be  always  important.  This  is  very  true  with  our 
engineering  and  technology  students  as  well.  In  my  opinion,  this  makes  it  difficult  to 
teach  irrelevant  content  without  validation  and  connection  to  the  world  external  to  each 
course  or  the  world  within  which  course  content  can  be  contextualized.  This  scenario 
changes  the  role  of  the  student  and  the  perspective  towards  education. 

Another  issue  has  risen  in  my  context,  one  where  when  students  are  working  and  cannot 
seem  to  make  the  demands  of  the  course  activities  work  with  their  employers’  demands 
and  schedules.  These  students  feel  the  professor  should  change  the  course  requirements 
so  they  are  easier  to  accomplish.  At  least  once  a semester  I have  this  discussion  with  my 
department  chair.  He  and  I always  end  up  agreeing  that  we  cannot  reduce  the  integrity  of 
the  course  or  its  required  activities.  This  does  not  mean  that  we  do  not  care.  We  do  work 
very  hard  to  understand  our  students'  lives  and  contexts  and  we  do  accommodate  them 
wherever  we  can,  but  course  content,  good  teaching  models  and  methods,  and  those  real- 
world  connections  have  an  integrity  base  of  their  own  - those  we  do  not  sacrifice. 
Therefore,  Gutek  provided  a great  opportunity  to  consider  her  transitions  with  those  of 
this  initiative. 

Good  teachers  have  these  discussions  all  the  time  and  are  always  rethinking  their  values, 
requirements,  strategies,  and  content.  It  is  exciting  to  do  so,  and  each  semester  brings  a 
new  and  different  context  to  the  table.  This  brings  the  perspective  of  our  public  into  the 
conversation  more  directly,  as  we  are  impacted  by  what  employers  want.  One  method  to 
include  a public  entity  is  through  course  content  validation,  in  our  case  through  business 
and  industry  or  our  students’  employers. 
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Approximately  every  two  to  three  years  or  so,  I send  my  eourse  syllabi,  with  additional 
information,  out  to  members  of  the  communities  of  practice  related  to  our  field  and 
courses.  They  validate  the  content,  and  often  we  also  discuss  the  teaching  methodologies 
apparent  in  the  requirements.  This  is  an  additional  way  of  seeking  course  content  than 
those  mentioned  above.  The  process  provides  great  feedback,  input  on  what  might  need 
to  be  added,  and  can  be  a way  of  prioritizing  or  weighting  the  significance  of  content. 
When  considering  the  public,  new  questions  become  part  of  the  equation:  "Who  is  our 
customer?"  and  "What  is  our  product?" 

Those  of  us  immersed  in  the  scholarship  of  teaching  debate  - whether  naturally,  or 
intentionally  - have  always  seen  research  and  teaching  as  equal,  connected,  and 
stimulating  roles  or  functions  within  the  professoriate.  We  are  always  changing,  at  least 
partially  driven  by  student  needs  and  expectations.  The  educational  and  learning  context 
for  students  is  complex  at  best.  In  their  context,  program  requirements,  scheduling 
conflicts,  educational  costs,  working  while  enrolled,  and  family  responsibility  can 
overwhelm  them.  This  has  changed  student  focus,  so  often  they  are  not  really  interested 
in  faculty  expertise  content.  Gutek  (as  cited  in  Andre  & Frost,  1996)  mentions  that 
"students  have  come  to  expect  a uniform  product  from  each  faculty  member  teaching  a 
class,  and  they  seem  to  have  little  knowledge  about  or  interest  in  special  areas  of 
expertise  of  the  faculty"  (p.  35).  Although  I agree  with  this,  I have  found  that  my  students 
are  interested  in  learning  just  about  anything  I want  them  to  learn  about.  However,  I must 
admit  that  interest  is  impacted  positively  by  the  ways  they  are  required  to  learn  about  it. 
One  must  risk  student  criticism  and  the  possibility  of  a negative  impact  on  course 
evaluations. 

For  those  professors  who  are  themselves  inquisitive  and  open  (who  are  learners  and 
seekers),  a natural  learning  community  develops  in  their  course  environments, 
microcosms  where  faculty  and  students  learn  from  each  other  as  well  as  from  the  course 
content  through  a variety  of  methods  and  processes.  Gutek  and  I have  a lot  in  common, 
and  both  of  us  have  learned  through  this  process  that  research  is  not  the  only  content 
response  for  course  content  but  is  important  as  one  source  of  content.  When  Gutek 
moved  from  psychology  to  business,  she  become  more  aware  that  there  is  a great  mixture 
of  content  important  for  courses,  especially  linking  theoretical  and  conceptual  to  its 
relevance  within  real  "communities  of  practice"  and  that  how  it  is  taught  truly  and  greatly 
impacts  what  is  learned.  Finally,  the  external  communities  of  practice  are  creating  new 
theory  and  concepts  that  have  yet  to  reach  the  research  base  or  textbook  content. 

She  also  describes  that  impact  on  higher  education  in  her  encounter  theory,  where  the 
relationship  between  provider  and  a recipient  of  goods  is  changing  to  encounters.  She 
goes  on  to  describe  "that  higher  education  systems  are  increasingly  moving  in  the 
direction  of  encounter  systems  to  save  costs  and  to  compete  to  provide  education 
services.  Encounter  systems  offer  uniform  products,  which  leads  to  the  point  that 

undergraduate  students  view  faculty  as  interchangeable  providers  of  service  rather 
than  as  unique  repositories  of  knowledge  and  ability  to  the  extent  that  they 
evaluate  delivery  process  as  more  important  than  content  of  material  covered. 
Higher  education  is  expected  to  resemble  mass  production  of  services 
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characteristic  of  encounter  systems  where  products  or  processes  are  uniform,  (p. 
38) 

I agree  that  this  is  bad  news  for  both  faeulty  and  students;  for  example,  general  edueation 
eourses  with  high  numbers  of  students  begin  to  resemble  eneounter  systems:  mass 
production  with  predictable  eontent  and  processes  regardless  of  who  provides  them.  All 
said  and  done,  Gutek  still  feels  that  teaching  and  research  are  interdependent  rather  than 
independent  ventures,  roles,  or  funetions,  and  I agree.  However,  we  must  realize  that 
universities  are  not  the  only  primary  produeers  of  researeh  and  even  the  university  work 
is  sometimes  not  published  as  quiekly  as  it  should  be.  Thus  in  my  opinion,  any  aecess  to 
research  only  provides  a deeper  and  broader  base  for  student  learning.  We  have  branched 
off  into  research  as  content  of  eoursework,  but  our  eonsideration  of  researeh  here  is 
multifaeeted:  researeh  as  eourse  eontent,  the  relationship  between  researeh  and  teaehing, 
and  further  to  faculty  endeavors  to  engage  in  researeh  on  teaching  and  learning  alongside 
their  disciplinary  research. 

Andre,  also  in  Andre  and  Frost  (1996),  eontends 

that  academics  do  not  do  teaching  and  research. . .at  least  [hopefully]  not.  These 
are  superficial  conceptualizations  of  our  professional  selves,  of  our  thoughts  and 
ideas  and  skills.  No,  what  we  do  is  deeper  and  more  eomplex:  we  seareh  for  truth, 
we  push  for  intelleetual  innovation,  we  share  our  truths  and  innovations  with 
particular  audiences,  and  we  create  environments  that  enhance  learning.  This  is 
much  more  exciting  and  more  involving  and  more  relevant  than  mere  teaching 
and  research. . . .Of  course  the  words  teaehing  and  researeh  do  eonvey  a rough 
sense  of  our  primary  audienees  (we  teaeh  our  students,  we  researeh  for  our  peers). 
Second,  the  words  are  a convenient  shorthand  for  our  behavior. . .what  we  do  on  a 
daily  basis  is  stand  up  before  audienees  and  profess  (we  teaeh),  and  then  we  spend 
quiet  time  diseovering,  organizing,  and  eommitting  our  ideas  to  print  (we  do 
researeh).  Third,  we  believe  that  success  in  teaching  and  research  is  measurable, 
(p.  41) 

Andre  diseusses  the  tired  issues  within  the  debate  but  also  brings  a different  perspeetive. 
She  describes  the  dichotomy  of  research-teaching  as  one  that  minimizes  what  professors 
do  when  eonsidering  complexity,  ideals,  and  impact,  making  the  point  that  it  is  mere 
eategorization.  This,  she  suggests,  is  eaused  by  our  failure  to  eommunieate  the 
eomplexities  of  what  we  aetually  do  to  the  external  world.  She  takes  the  debate  to  a 
different  plane: 

Words  eonnote  an  unfortunate  status  bias,  with  teaehers  and  teaching  being 
ranked  below  researchers  and  writing.  Culturally,  the  upper  elass  thinks,  reads, 
and  writes,  but  does  not  teaeh,  as  that  may  be  below  their  station;  they  produee 
intellectual  [original  or  extensions]  works.  Among  those  elasses,  leisure  is  more 
desirable  than  work  or  scheduled  activity-work,  e.g.  scheduled  by  an  institution. 
Therefore,  in  our  eontext,  researeh  is  self-seheduled,  versus  teaehing  seheduled  by 
others.  Researeh  is  ereating  with  freedom  and  individuality  in  taet,  whereas 
teaching  may  be  perceived  presenting  “someone  else's  ideas”  suggests  servitude 
and  conformity,  (p.  41) 
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Andre  presents  thoughts  that  this  dichotomy  reveals  who  we  become  and  how  we 
succeed  in  the  profession  and  that  the  commonality  between  the  two,  teaching  and 
research,  is  "striving  for  truth  and  innovation  in  our  field(s). . .addressing  different 
audiences  through  different  modes”  (p.  42). 

Andre  defines  truth  and  innovation.  As  scholars,  truth  may  be  exemplified  in  the 
research  methodology  we  choose  or  a personal  predilection  of  reality.  She  mentions  that 
one  way  academics  see  truth  is  through  a peer  review  system,  but  business  schools  see  it 
more  from  the  context  of  practitioners  within  a community  of  practice  or  field.  In  other 
words,  there  are  many  different  forms  of  truth.  What  is  true  for  one  may  not  be  true  for 
another;  thus  perhaps  it  is  our  responsibility  to  provide  students  with  contextual  truth. 
Consider  the  difference  between  teaching  students  to  critique  business  practices  and 
merely  learning  business  practices.  One  teaches  students  how  to  think  critically  and  learn 
through  that  process;  the  other  merely  requires  memorizing  or  rote  learning.  What  we 
teach  is  important,  but  how  we  teach  is  even  more  so.  In  the  latter  portion  of  the  above 
example,  we  are  molding  student  visions  of  the  world  and  developing  their  capability  to 
transform  the  world. 

Andre  discusses  what  innovation,  audience,  and  learning  means  to  her.  In  her  discussion 
on  innovation,  she  discusses  the  sad  state  of  affairs  where  publishing  for  publishing  sake 
or  using  publishing  as  a method  to  determine  the  pecking  order  of  professionals  or  the 
most  active  "thinker"  in  the  field  is  just  not  of  interest.  She  is  more  concerned  about  the 
quality  of  research  and  being  candid  about  whether  a new  idea  is  really  a new  idea  rather 
than  new  terminology  or  more  modern  examples  of  earlier  thinking.  She  is  not  rejecting 
research  but  rather  calling  for  us  to  rethink  what  research  would  be  useful  or  responsible: 
what  is  new  versus  what  is  extended  into  or  uses  in  new  contexts.  In  Senge's  (1990) 
terminology,  the  professionals  may  not  actually  be  "generative,"  which  is  not  necessarily 
bad  if  they  are  involved  in  other  areas  of  scholarship  that  could  move  more  toward 
interdisciplinary  thinking.  Gutek  feels  that  research  needs  to  have  broader  impact  on  each 
other  as  academics; 

meanwhile,  we  should  honor  the  knowledge  base  that  already  exists,  admitting 
that  we  already  know  some  important  stuff  that  isn't  our  own.  We  should  assert 
that  applying  already  existing  theories  to  real-world  problems  may  be  just  as  [or 
more]  important  than  putting  our  name  on  a new  theory. . . .This  assertion  would 
make  our  field  more  relevant  and  perhaps  more  understood  by  the  public,  (as  cited 
in  Andre  and  Frost,  1996,  p.  50) 

In  examining  audience(s),  she  indicates  that  she  is  her  first  audience,  then  her  students, 
followed  by  her  colleagues  where  we  teach  each  other  (or  should).  Ideally  we  are  a 
community  of  scholars,  a community  of  learners,  but  colleagues  seldom  learn  from  each 
other.  To  learn  from  each  other, 

we  have  to  admit  to  our  own  ignorance,  and  this  is  hard  to  do  within  systems  that 
value  knowing  rather  than  learning,  that  celebrate  experts  rather  than  interactive 
teams,  that  are  based  on  politics  rather  than  authentic  process. . . [So]  we  continue 
to  reward  what  is  measurable  and  to  miss  the  interesting  academic  dialogue  we 
yearn  for. . . we  do  not  practice  what  we  preach. . . .and  the  academic  audience 
defines  truth  [where  she  began]  rather  narrowly  and  arbitrarily... [finally] 
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remember  understanding  business  is  erueial  to  our  profession. . . [but]  we  have  to 
be  eareful  about  making  business  our  target  audience,  (p.  51) 

I propose  that  the  most  important  gift  we  can  give  each  other  is  a comfort  zone  where  we 
are  able  to  be  open  and  candid  about  what  we  do  not  know,  trusting  each  other  enough  to 
be  honest.  That  will  move  collaboration  forward  as  much  as  anything  else  a researcher  or 
project  director  can  do  - if  they  are  able  to  gain  buy  in  from  the  group.  The  professor 
group  we  have  worked  with  seemed  fairly  comfortable  throughout  the  faculty 
development  program,  openly  admitting  what  they  do  not  know  about  or  do.  During  the 
analysis  segment,  they  openly  admitted  what  teaching/leaming  models,  strategies, 
techniques  or  procedures  they  were  not  accustomed  to  using  or  had  not  tried.  This  in 
itself  is  the  most  positive  foundation  to  build  when  developing  faculty  groups. 

Gutek  (as  cited  in  Andre  & Frost,  1996)  feels  that  we  are  not  preparing  students  for  a job 
or  a career  but  rather  to  understand  society  and  that  she  is  not  educating  them  to  fit  in  but 
rather  to  understand  their  position  in  the  world  of  organizations.  This  perspective, 
although  she  is  in  a professional  school  of  business  possibly  analogous  to  an  engineering 
and  technology  school  may  be  argued.  I agree  we  are  giving  students  today's  knowledge, 
methods,  techniques,  tools  when,  more  importantly,  we  should  be  situating  our  students 
to  be  generative  (Senge,  1990).  When  she  discusses  learning,  she  writes  of  the  same 
questions  that  have  often  come  to  me:  "Why  are  you  changing  your  courses  so  often?"  or 
from  the  outside  world  "How  can  you  stand  to  teach  the  same  courses  over  again?"  My 
response  was  almost  identical  to  hers:  "They  are  not  the  same;  there  is  new  content, 
different  methods,  and/or  new  issues  each  time  I teach  them;  they  have  to  change!  It  is 
not  boring  - but  the  other  part  of  my  answer  that  I do  not  always  express  is  that  I have 
learned  so  much  during  and  between  each  time  of  teaching  each  course;  my  research, 
travel,  reading,  conferences,  colleagues,  and  very  often  my  students,  especially  those  who 
work,  teach  me  so  much  in  so  many  ways.  It  may  not  be  always  new  knowledge,  but  may 
be  the  context  within  it  is  applied  or  employed  to  solve  a problem,  and  so  much  more. 
Both  of  us  truly  learn  from  everyone  and  every  situation;  remember  always  that  one's 
existing  knowledge  is  "a  gift  that  keeps  on  giving,"  in  that  each  time  it  is  used,  it  expands 
and  deepens,  becoming  greater  and  different  than  it  was  and  evolving  into  new 
knowledge.  IF  we  are  learning  individuals  and  not  merely  the  "learned,"  there  is  hope  for 
academics. 

Frost,  also  in  Andre  and  Frost  (1996),  explores  the  academic  credibility  of  teaching  and 
thinks  that  an  open  mind  about  research  or  teaching  along  with  an  expectation  of 
creating,  discovering,  improving,  and  generating  new  understandings  when  doing  either 
promotes  learning  as  ongoing.  Thus  it  is  possible  for  both  roles  and  functions  to  feed  and 
enhance  the  other.  Additionally,  Mahoney  identifies  themes  on  teaching  as  research  and 
learning: 

1 . a focus  on  analytic  theory,  rather  than  on  description  or  presentation  of  best 
practices 

2.  an  attempt  to  develop  students'  abilities  to  think  and  employ  multiple 
theoretical  models  from  multiple  disciplines. . .that  rarely  can  a single 
discipline  provide  total  and  complete  understanding  of  a phenomenon  or 
issue... use  multiple  disciplinary  lenses  typically  enhances  understanding 
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3.  an  implicit  theme  is  a focus  on  a form  of  what  might  be  termed  a Socratic  method 
of  learning. . . seek[ing]  our  conundrums  and  contradictions . . . [e.g.]  focus  on 
anomalies  and  use  of  multiple  conceptual  approaches,  (as  cited  in  Andre  & Frost 
pp.  118-119) 

Mahoney  would  prefer  that  students  have  required  studies  across  disciplines  for  breadth, 
which  would  stimulate  understanding  and  interest.  Most  importantly,  and  in  this  I 
adamantly  agree,  this  multifaceted  approach  to  teaching  and  learning  also  furthers  the 
professors’  learning  along  with  the  students.  They  discover,  explore,  argue,  and 
ultimately  learn  together  - my  most  preferred  type  of  interaction  with  my  students.  As 
students  discover,  and  we  along  with  them,  perhaps  this  is  another  type  of  research  lens? 
For  Mahoney,  research  is  all  about  unanswered  questions  or  issues  or  contradictions  that 
demand  explanation.  This  can  happen  with  one's  own  research  but  also  with  student 
research  at  all  levels,  undergraduate  and  the  graduate.  Mahoney  wraps  his  thoughts  with 
the  usual  conceptualization  of  teaching  and  research  as  separate  but  joined 
endeavors  in  academic  scholarship  would  have  blinded  me  to  what  I now  realize 
is  an  inevitable  joining  of  them  in  a career  of  learning.  The  learning  I thrill  to  and 
enjoy  is  not  possible  through  teaching  or  research  as  a separate  endeavor.  And  I 
have  come  to  realize  that  the  teaching  I enjoy  involves  research;  it  stimulates  and 
requires  research.  Similarly,  the  research  I enjoy  would  be  sterile  without  the 
accompaniment  of  teaching.  For  me,  teaching  and  research  are  but  different 
emphases  of  the  same  process,  (p.  124) 

Moving  more  deeply  into  and  forward  in  time  beyond  the  1990  discussions,  arguments, 
and  calls  for  action  to  more  clearly  define  professoriate  roles,  functions,  and 
responsibilities  and  the  rewards  for  particular  aspects  of  their  work,  Ruben  (2004) 
addresses  the  discussion  and  arguments  from  a quality  perspective:  Pursuing  Excellence 
in  Higher  Education.  Ward's  introduction  to  this  volume  sets  the  tone  for  the  content  and 
context  as  change  and  the  higher  education  responses  to  change  and  how 

change  itself  has  created  tensions  about  the  mission  and  values  of  higher 
education.... Some  are  healthy  and  creative;  others  are  threatening  and  destructive 
. . . [and]  are  rooted  in  the  interplay  of  an  historic  public  mission  and  new  market- 
based  strategies. . . .[as]  public  investments  in  higher  education  have  yielded  public 
good,  but  increasingly  resource  allocations  designed  to  serve  that  public  good  are 
based  on  market  conditions. . .different  but  not  necessarily  irreconcilable  values. 

(p.  xi) 

Ward  sets  the  context  for  the  discussion  about  changes  in  higher  education  over  the  past 
decade  as  one  where  the  changes  are  greater  in  scope  and  magnitude  than  any  since  the 
1960s.  In  describing  the  shifts  in  how  higher  education  is  funded  since  WWII  (e.g.,  the 
growing  dependence  upon  tuition,  endowments,  and  reduced  state  support),  the  targeted 
growth  of  federal  research  support,  etc.  has  redefined  leadership  and  management 
requirements.  Higher  education  institutions  have  not  been  clear  about  their  missions, 
which  combined  with  market  forces  (public  pressure  to  improve  the  undergraduate 
learning  experience,  changes  in  revenue  sources,  financial  aid,  student  access  to 
education,  leadership  and  management  strategies,  and  the  growth  and  success  of  private 
for  profit  education  providers)  have  greatly  changed  the  operational  context  of  higher 
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education.  Critical  also  is  the  challenge  that  information  technology  brings  for  the 
development  of  new  pedagogies,  distance  learning,  and  scheduling  - all  with  competition 
issues. 

There  is  a powerful  market  of  alternative  providers,  so  capacity  has  become  an  important 
issue.  Although  this  is  not  an  issue  for  us  because  we  have  long  had  excellent  articulation 
with  two-year  institutions  and  value  transfer  students  at  the  general  education  or  major 
level  and/or  those  who  are  "spiraling"  and  are  enrolled  at  more  than  one  institution 
simultaneously,  this  has  to  be  carefully  monitored  so  that  students  do  not  take 
unnecessary  courses.  These  situations  require  high  quality  advising  and  the  coordination 
of  that  advising  across  institutions  - a complex  problem  for  each  institution  and  the 
students.  Also  the  surge  of  demand  by  first  generation  students  has  grown  in  densely 
populated  areas,  which  has  dramatically  changed  our  context  and  creates  quality  control, 
funding,  leadership,  and  management  issues  requiring  nontraditional  responses  - creative 
and  often  "out  of  the  box"  responses  and,  ultimately,  a new  type  of  institutional 
leadership.  We  have  always  had  a variety  of  choices  for  higher  education  in  America; 
however,  more  choices  are  becoming  available,  and  institutions  are  beginning  to  define 
what  they  consider  their  "niche"  in  the  overall  scheme  of  choices.  "This  more  highly 
differentiated  range  of  higher  education  opportunities  may  be  related  to  a more 
competitive  market  environment  for  resources  and  students,  but  concerns  about  access 
and  quality  remain  part  of  a set  of  enduring  values"  (p.  xiv).  This  transformation  is  in 
process,  relatively  undocumented,  incomplete,  and  global  in  scale  and  context. 

Higher  education  is  an  international  currency.  We  are  unique  in  public-private 
partnerships  and  the  enormous  scale  that  has  defined  our  education.  We  have  an  interplay 
of  markets  and  missions  different  from  anywhere  in  the  world,  "expressed  primarily 
through  resource  reallocations  made  necessary  by  the  changing  demands  and  needs  of 
society”  (p.  xv).  Although  some  of  these  are  driven  by  the  market,  some  new  priorities 
are  also  driven  by  the  independent  efforts  to  enhance  or  redefine  the  mission  of  higher 
education  to  better  meet  the  changing  needs  of  students  and  to  better  engage  in  new  ways 
to  advance  knowledge.  Ruben  (2004)  ends  with: 

the  most  curious  aspect  of  our  role  in  the  advancement  of  knowledge  is  our  lack 
of  curiosity  about  higher  education  as  an  institution  and  its  place  in  society. . . 
Universities  are  not  only  places  of  learning  but  also  organizations  and  cultures 
with  their  own  deliberative  and  reactive  relationships  to  society  at  large. . .often 
judged  slow  to  change. . . [They]  should  also  be  an  arena  where  debate  about 
public  and  private  values. . .access  and  quality. . .tradition  and  change. . .market  and 
value  driven  decision  making  are  fully  and  openly  explored,  (p.  xv) 

Ruben  (2004)  describes  the  academy's  response  to  the  criticism  from  the  1990s  and 
forward  as  mixed;  some  have  ignored  it,  considering  their  institutions  and  positions 
historically  controversial  with  a mission  of  "holding  up  a mirror  to  society  and 
challenging  conventional  ways"  (p.  3).  To  further  reflect  this  point,  he  quotes  Carey  in 
The  Engaged  Discipline: 

Contemporary  academics  are  often  embarrassed  and  defensive  about  the  invidious 
contrast  between  the  academy  and  the  'real  world'. . .1  take  that  distinction  as  a 
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tribute,  for  the  relevant  contrast  is  not  between  the  real  and  the  imitation  but 
between  the  sacred  and  the  profane.  The  gates  of  the  university  mark  a passage 
not  only  from  the  city  to  the  campus  but  from  the  vulgar  and  ordinary  to  the 
hallowed  and  unique.  (Carey,  2000,  as  cited  in  Ruben,  p.6) 

What  are  the  right  questions,  if  considering  that  the  public  does  not  quite  understand 
higher  education's  mission?  With  understanding  and  the  right  facts  available,  would  the 
public's  perspective  change?  Many  academics  find  the  discord  about  their  work,  their  role 
and  function,  and  their  mission  (or  what  they  think  their  mission  is)  disheartening  and 
demoralizing  - also  difficult  to  ignore.  Along  with  the  economic  pressures  of  today,  K-12 
and  other  organizations  are  competing  for  scarce  resources  in  an  intense  environment. 
Therefore,  many  have  concluded  that  we  have  no  choice  but  to  adapt  and  change.  We 
must  not  risk  obsolescence  and  atrophy.  Noble  (2001)  is  passionate  about  these  issues 
and  argues  vehemently  about  the  commercialization  of  intellectual  property  and 

the  need  to  reaffirm  the  traditional  ideas  of  academic  purpose  and  promise. . .and 
to  recapture  the  ideological,  rhetorical,  and  political  initiative  and  the  moral  high 
ground  in  the  debates  about  higher  education  in  order  to  reinvigorate  a 
noncommercial  conception  of  higher  education  and  to  re-consecrate  the  intrinsic 
rather  than  the  mere  value  of  universities . . . [and]  that  faculty  represent  the  last 
line  of  defense  against  the  wholesale  commercialization  of  academia,  of  which  the 
commodifcation  of  instruction  is  just  the  latest  manifestation,  (p.  32) 

I agree,  but  in  my  opinion,  change  must  be  purposeful  with  intentional  outcomes 
identified.  Ruben  (2004,  p.7)  identifies  the  critical  challenges  and  thoroughly  explores 
each  one: 

• Broadening  public  appreciation  for  the  work  of  the  academy 

• Increasing  our  understanding  of  the  needs  of  the  workplaces 

• Becoming  more  effective  learning  organizations 
-goal  clarification 

-supportive  and  facilitative  processes 

-receptivity  to  a range  of  information  and  information  sources 

• Integrating  assessment,  planning,  and  improvement 
-to  identify  criteria  for  organizational  effectiveness 
-to  recognize  leading  organizations 

-to  promote  dissemination  of  effective  practices 

• Enhancing  collaboration  and  community  [within  and  between  faculty  and  staff] 
Those  who  work  in  academe  inhabit  an  unofficial,  yet  undeniable  caste  system. 
Tenured  Ph.D.s  constitute  the  Brahmin  etc.,  followed  by  untenured  faculty  and 
staff  members,  and  research  associates,  librarians,  secretaries,  food-service 
personnel,  and  finally,  the  untouchables;  physical  plan  employees.  (p.l9) 

• Recognizing  that  everyone  in  the  institution  is  a teacher 

• Devoting  more  attention  and  resources  to  leadership 

• More  broadly  framing  our  vision  of  excellence 
-academic  excellence 

-service  excellence 
-operational  excellence 
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Ruben  (2004)  notes  that  to  move  beyond  the  "ivory  tower,  perhaps  the  academy  should 
continue  to  protect  that  environment  described  above,  but  also  genuinely  respond  to  the 
concerns  of  its  external  constituencies,  sustain  its  virtue,  and  address  the  necessary  by 
creating  a new  and  more  encompassing  vision  of  excellence  that  takes  account  of 
opposing  views  of  higher  education's  purpose  while  underscoring  the  importance, 
interdependence,  and  useful  tensions  among  the  goals  of  the  academy:  academic,  service, 
and  operational  excellence.  In  addressing  teaching  and  learning,  he  uses  a framework  that 
identifies  two  components:  1)  courses  and  programs  and  2)  student  outcomes.  He 
presents  multiple  dimensions  to  consider  when  evaluating  the  quality  of  courses, 
programs,  and  student  outcomes  from  the  perspective  of  many  different  groups  (e.g. 
colleagues,  students,  alumni,  employers,  etc.)  and  offers  a range  of  indicators  from  which 
to  judge  success.  His  final  chapter  presents  a description  of  a group  striving  to  stimulate 
and  monitor  changes  in  higher  education.  In  "Taking  Charge  of  Change:  The  Kellogg 
Commission  on  the  Future  of  State  and  Land-Grant  Universities,"  Byrne  reviews  the 
agenda  for  1996  as  1)  the  student  experience;  2)  student  access;  3)  the  engaged 
institution;  4)  a learning  society;  5)  campus  culture;  and  6)  the  covenant  or  partnership 
between  the  public  and  the  public's  universities.  Their  process  involved  focus  groups  for 
each  agenda  item.  Each  group  has  issued  reports  with  recommendations  (40)  and 
suggested  actions  (70)  for  reform.  These  groups  were  followed  by  the  creation  of  regional 
groups  to  further  stimulate  and  support  reform.  Then,  in  2000,  the  group  administered  a 
survey  on  reform  at  36  public  universities,  addressing  items  1-6.  Responses  were  sought 
from  four  levels  at  each  institution.  They  received  90  responses  from  the  36  institutions 
(62.5%  -36  institutions  x 4 levels  of  responses  = 144  possible  responses,  a 62.5%  return). 
The  responses  were  reported  as  "reflecting  the  progress  of  reform  occurring  today  in  the 
public  university"  (p.  363).  A post-commission  workshop  "A  vision  for  change"  was 
conducted  on  the  requirements  for  changing  in  2001.  The  Commission's  work  has 
impacted  American  public  education,  and  its  reports  have  enhanced  the  awareness  for  the 
need  for  reform.  "The  integration  of  learning,  discovery,  and  engagement  as 
recommended  by  the  Commission  is  being  adopted  as  the  core  mission  at  an  increasing 
number  of  institutions"  (p.  364). 

Values. 

By  now,  those  reading  this  are  beginning  to  grasp  the  predicament  within  which  today's 
higher  education  finds  itself  and  the  context  within  which  the  issues  about  teaching  and 
research  exist.  However  to  make  sure  that  I have  "beaten  the  dead  horse,"  I will  extend  it 
to  include  some  thoughts  from  others  who  have  written  on  the  topic  of  today's  higher 
education  and  its  inherent  dilemmas.  Kennedy  (1997),  once  president  of  Stanford 
University,  approached  the  topic  from  the  perspective  of  academic  duty.  He  believes  that 
the  modern  American  university,  with  all  of  its  warts,  is  a "real  triumph"  and  that  those 
who  are  critical  offer  no  tangible  diagnosis  or  suggestions  for  a cure"  (p.  vii).  He  writes 
for  and  about  members  of  the  faculty  regarding  their  central  roles  in  the  institution's 
mission  and  to  students.  In  his  opinion,  academic  duty  is  the  counterpart  to  academic 
freedom.  His  discussion  centers  on  the  fact  that  faculty  work  is  "mysterious  to  those 
outside"  and  that  perceived  missing  information  is  mistaken  for  lack  of  accountability. 

His  comment  to  future  professors  is 


22 


that  you  are  entering  a life  full  of  the  most  interesting  challenges  - and  the  most 
important  mission  that  can  be  found  in  a modern  society.  The  university  is  above 
all  else  about  opportunity:  the  opportunity  to  give  others  the  personal  and 
intellectual  platform  they  need  to  advance  the  culture,  to  preserve  life,  and  to 
guarantee  a sustainable  human  future.  Could  anything  possibly  matter  more  than 
that?  (p.  viii) 

This  description  especially  resonates  with  me  and  with  the  fields  of  engineering  and 
engineering  technology.  When  considering  Kennedy’s  (1997)  comments  in  light  of  a 
definition  that  I prefer  for  technology:  "the  science  of  efficient  action  that  extends  human 
potential  and  capacity,"  I now  add  “that  extends  individual  power,  thus  the  collective 
power  of  humanity.”  His  description  of  higher  education  seems  to  fit  well  as  the  mission 
to  prepare  students  to  research,  invent,  or  develop  knowledge  or  technological  advances 
that  in  turn  "advance  culture,  preserve  life,  and  guarantee  a sustainable  human  future." 
(Snyder  & Hales,  1981) 

The  concept  of  "academic  freedom"  is  misused.  Its  origin  was  a result  of  the  anti- 
communist sentiment  after  WWII.  At  that  time.  Congress  put  pressure  on  universities  to 
fire  faculty  for  past  membership  in  any  organization  deemed  un-American  or  sympathetic 
to  Communism.  Universities  resisted  that  to  varying  degrees,  but  the  tradition  of 
academic  freedom  strengthened  their  capacity  to  do  so.  Academic  freedom  creates 

a treasured  space  for  intellectual  experiment  - treasured  in  part  because  it  is  safe 
[to  experiment] . In  practice  such  freedom  extends  further,  permitting  unusually 
creative  people  to  lead  unusually  creative  lives.  Indeed,  academic  freedom 
connotes  loose  structure  and  minimal  interference. . .no  time  clocks  and  few 
regulations  about  the  direction  of  effort  or  even  about  the  location  at  which  it  is  to 
take  place.  So  distinct  is  the  academy  from  other  workplaces  that  we  have 
developed  an  informal  vocabulary  to  describe  its  separation: . . .the  ivory  tower, 

. . .everything  else  the  real  world,  (pp.  1-2) 

Its  counterpart,  academic  duty  is  rarely  thought  about:  opposite  sides  of  the  same  coin. 
Gardner,  in  Kennedy  (1997),  said  "of  the  symmetry  between  individual  freedom  and 
communitarian  obligation,  'Liberty  and  duty,  freedom  and  responsibility:  that's  the  deal'" 
(p.  2).  So,  the  paradox:  American  education  has  never  been  more  successful;  more 
Americans  are  receiving  higher  education  than  ever  before,  serving  more  people  than 
ever.  Institutions  and  faculty  are  highly  respected.  More  people  are  paying  for  their 
children's  education.  It  is  still  strong;  international  students  come  to  us  from  across  the 
globe.  We  have  the  strongest  university-based  research  system  in  the  world.  It  is 
considered  an  innovation  incubator  important  to  economic  development.  However,  public 
criticism  is  at  the  assault  level,  coming  from  both  the  Left  and  Right.  We  are  perceived  as 
failing  in  science  and  policy  studies  (e.g.,  an  AIDS  cure,  a failing  K-12  educational 
system,  lack  of  English  speaking  teachers  or  professors,  corporations  engaged  in  eternal 
downsizing,  and  productivity  in  higher  education).  This  has  caused  a serious  morale  issue 
in  the  academy.  What  Kennedy  says  about  this  is  important;  there  seems  to  be  a 
dissonance  between  what  the  public  sees  as  our  purpose  and  what  the  academy  or 
university  sees  as  its  purpose.  The  freedoms  mentioned  above,  necessary  to  teaching  and 
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scholarly  work,  are  understood  and  accepted  rather  well;  however,  the  counterbalance, 
that  of  duty  or  obligations  are 

vague  and  obscure.  Put  baldly,  there  is  confusion  about  what  is  owed:  by  the 
university  to  society,  by  faculty  to  students,  by  administrators  to  both.  Academic 
freedom  is  a widely  shared  value;  academic  duty,  which  ought  to  count  for  as 
much,  is  “mysterious... [and]  no  less  a mystery  within  the  walls  of  the  ivory 
tower. . . [with]  little  [being]  said  to  new  faculty  members . . . about  the  nature  of 
faculty  responsibilities... No  job  descriptions,  no  annual  performance  evaluations 
[unless  on  merit  system  as  we  are],. . .The  expectations  of  the  professoriate  are 
murky,  and  public  understanding,  murkier  still  (pp.  2-3). 

The  burden  for  higher  education  is  greater  than  ever  before;  beyond  developing  students 
into  more  knowledgeable,  cultured,  skilled,  and  thoughtful  individuals,  there  are  now 
additional  and  serious  expectations  that  universities  undertake:  economic  development, 
international  competitiveness,  research  for  better  health  care,  and  military  preparedness. 
Higher  education  is  truly  woven  into  the  fabric  of  American  lives  and,  furthermore, 
expected  to  provide  cultural  inspiration  and  athletic  entertainment  on  weekends!  The  fact 
remains  that  when  it  fails,  we  are  disappointed,  and  when  it  is  expensive,  we  become 
angry.  Kennedy  (1997)  feels  that  the  academy  is  an  extraordinary  institution  but 
questions  how  it  came  to  have  so  many  responsibilities  - full  of  paradoxes. 

The  different  ideas  or  perspectives  about  higher  education  are  partly  due  to  individual 
experiences,  but  also  because  each  institution  is  different.  With  over  3000  four-year 
institutions  in  the  U.S.,  there  are  similarities  and  great  differences  (different  missions), 
thus  the  public's  vast  array  of  perceptions.  Higher  education  is  perceived  as  a way  to 
increase  social  status;  some  view  it  as  elitist  and  "stuck  up,"  but  too  much  education  is 
viewed  by  some  as  suspicious  and  they  joke  about  the  absent-minded  professor. 
Regardless,  higher  education  is  a national  tradition,  both  admired  and  suspect;  it  is  more 
successful  than  ever.  Perceived  as  an  institution  to  produce  thinking  and  working 
graduates  with  the  ability  to  communicate  effectively,  analyze  problems,  and  be 
reflective  about  cultural,  some  see  it  as  simply  a credentialing  mechanism  for 
employment.  Others  see  it  as  both,  with  conflict  between  the  two.  There  are  curriculum 
content  collisions;  what  should  students  be  required  to  study,  great  works  or  great  ideas. 
Western  culture,  or  to  quote  Jesse  Jackson  and  demonstrators  "Western  culture's  got  to 
go"  (Kennedy,  1997,  p.  9).  Institutions  engaged  in  great  debate,  as  they  should,  and 
curricular  change  became  an  external  concern,  especially  in  light  of  the  relationship 
between  knowledge  and  values. 

Of  course,  the  way  institutions  are  governed  is  sometimes  suspect,  and  funding  issues 
have  a great  deal  to  do  with  the  intensity  of  the  criticism  by  the  public.  So  institutions 
have  become  good  at  lobbying  for  their  interests,  which  has  caused  them  to  be  perceived 
in  a different  light,  less  special  and  "just  another  interest  group"  (Kennedy,  1997,  p.  9). 
The  costs  of  the  educational  process  have  increased  dramatically,  but  when  considered  in 
light  of  "new  knowledge"  production,  each  new  unit  of  incremental  knowledge  costs 
more  than  those  before.  Perhaps  our  appetite  for  growth  (e.g.,  the  increase  in  scientific 
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literature  reporting  results)  needs  to  be  evaluated  when  considering  the  high  cost 
implications  for  universities. 

Other  perceptions,  not  often  thought  about,  come  into  play;  the  relationships  between 
public  universities  and  private  corporations  have  grown.  The  public  really  still  views  the 
professor  as  a bit  shabby  and  absent-minded,  but  that  contrast  between  the  ivory  tower 
and  public  is  broken  when  professors  begin  to  be  perceived  as  rich  and  clever.  Not  only 
does  the  public's  perspective  change  about  them  as  individuals,  but  also  towards  their 
institutions.  Some  feel  that  universities  should  not  patent  - that  faculty  members  have 
their  freedom,  are  paid  enough,  and  should  not  really  profit  from  their  research. 
"Academics  and  profitability  don't  mix"  (Kennedy,  1997,  p.  12).  On  the  positive  side  of 
the  coin,  in  1992,  survey  results,  published  by  the  national  Opinion  Research  Center, 
showed  that  Americans  expressed  the  greatest  confidence  in  the  leaders  of  medicine, 
science,  and  education;  the  levels  were  substantially  higher,  for  example,  than  for  the 
Congress,  labor,  the  press,  or  the  executive  branch  of  the  federal  government. 

Even  though  the  accomplishment  is  so  strong,  there  still  remains  a negative  public 
perspective  about  the  nature  of  the  modern  university.  He  summarizes  by  reminding  us 
that  universities  are  controversial  entities,  their  actions  draw  public  notice  and  scrutiny 
because  they  are  important.  Kennedy  (1997)  sees  today's  style  of  dealing  openly  and 
publicly  with  problems  of  the  academy  as  a healthy  sign,  but  also  feels  that  we  must  be 
careful  that  the  "publicizing"  of  issues  does  not  result  in  them  appearing  greater  than  they 
really  are.  Although  misunderstood  and  criticized,  the  fact  remains  that  society  needs 
them,  and  universities  need  the  public's  trust.  We  must  consider  the  criticism,  its  origin, 
and  come  to  grips  with  our  responsibilities,  clarify  our  duty,  regain  public  acceptance, 
ultimately  fulfilling  an  important  obligation  to  "the  society  that  nurtures  us"  (p.  22). 

The  very  heart  of  the  institution's  academic  duty  to  society  is  the  work  of  its 
faculty. . .Through  the  fulfillment  of  their  academic  duties,  they  fulfill  the 
institution's  duty  to  society. . ..It  can  truthfully  be  asserted  that  they  are  the 
institution.  Responsibility  suggests  the  duty  one  owes  to  the  institution  - and,  first 
and  foremost,  to  one's  students. . .classes  well  prepared,  giving  a student  time, 
remaining  . . . objective. . .delivering  a full  set  of  institutional  objectives. . .includes, 
but  is  not  limited  to,  professional  ethics,  (p.  15) 

Kennedy  (1997)  thoughtfully  and  more  deeply  considers  the  issues  throughout  his 
chapters  on  "preparation,"  focusing  on  prestige  perceived  by  the  university  and  prestige 
perceived  by  the  public  and  how  the  division  of  labor  between  teaching  and  research  is 
affecting  the  quality  of  life  for  many  professors,  with  an  environment  of  faculty  members 
being  told  to  concentrate  on  research  or,  if  necessary,  skimp  on  research,  while  also 
experiencing  heavier  teaching  loads  due  to  financial  constraints.  Even  though  the  most 
realistic  measure  of  institution  prestige  is  faculty  quality,  there  are  no  consistent  criteria, 
as  each  institution  is  different.  Eaculty  members  have  begun  to  consider  other  factors: 
location,  careers  for  spouses,  facilities,  valued  colleagues,  the  research  versus  teaching 
responsibility,  and  salary,  meaning  that  we  cannot  really  describe  a typical  career  across 
institutions  or  disciplines.  Institutional  prestige  is  a powerful  and  conservative  concept; 
however,  some  aspect  of  it  is  an  illusion.  Eor  example,  a Harvard  graduate  will  not 
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always  work  at  Harvard,  so  does  that  mean  that  they  no  longer  have  prestige  when 
employed  at  a "less  prestigious"  institution?  Harvard,  Yale,  and  Stanford  send  professors 
across  the  nation;  perhaps  their  value  system  needs  adjustment.  Another  way  of  looking 
at  it  locally  is  reflected  by  a professor  commenting  that  he  should  not  have  to  teach  some 
of  the  students  he  has;  after  all  he  has  a degree  from  ****.  Yet  another  way  of  looking 
at  it  occurred  at  a major  university  committee  meeting;  some  liberal  arts  and  science 
professors  were  complaining  about  the  quality  of  our  transfer  students  and  describing 
their  preparation  by  the  community  colleges  as  deficient.  Our  associate  provost  at  the 
time  quickly  said,  "Careful,  many  of  the  instructors  and  professors  have  our  (NIU) 
degrees  - what  do  you  have  to  say  to  that?"  There  are  so  many  levels  from  which  to 
consider  prestige  that  Kennedy  calls  for  a "dose  of  reality"  (p.  58).  Research  I universities 
are  a small  minority  of  the  higher  education  community. 

Kennedy  (1997)  goes  into  the  duty  of  teaching  as  the  very  core  of  the  university's  mission 
and  the  faculty's  academic  duty,  discussing  how  exciting  and  experimental  the  best 
teaching  has  become.  New  strategies  are  being  implemented,  and  technology  is 
revolutionizing  the  faculty-student  relationship,  perhaps  resulting  in  a positive  outcome 
of  accelerating  the  more  routine  work  of  learning,  while  reserving  for  the  faculty 
members  the  most  meaningful  forms  of  interaction  with  students.  However,  for  young  or 
new  faculty  members,  we  cannot  yet  assure  them  that  teaching  is  a priority  equal  to 
research,  but  "times  are  changing.  The  day  may  not  be  far  off  when  teaching 
performances  are  routinely  reviewed  by  peers,  when  senior  academic  visitors  conduct 
teaching  ‘master  classes’  as  well  as  give  research  seminars,  and  [the  ultimate]  when 
candidates  are  told  that  teaching  is  important  by  department  chairs  who  really  mean  it[!]” 
(p.  96). 

A direct  example  occurred  here  at  NIU  in  the  1980s.  A presidential  award  was  created  to 
recognize  major  researchers  on  campus;  therefore,  a committee  worked  with  the 
provost’s  and  president's  offices  to  create  the  award,  criteria,  and  nomination  process  for 
the  NIU  Presidential  Research  Award.  This  caused  some  academic  scandal  on  campus,  as 
those  who  did  not  fit  the  pure  or  basic  research  criteria  did  not  fit  into  the  traditional 
liberal  and  sciences  framework  or  they  were  engaged  in  the  other  types  of  research 
described  by  Boyer  (1990).  There  was  no  acknowledgement  of,  or  value  for,  excellence 
in  teaching  or  scholarship  in  teaching;  therefore,  the  committee  for  the  improvement  of 
undergraduate  education  submitted  an  argument  for  creating  an  NIU  Presidential 
Teaching  Award  to  the  President.  Thus,  we  have  an  equal  award  in  all  ways,  unless 
considering  individual  perspectives  on  the  priority  of  research  versus  teaching.  The 
creation  of  these  awards  was  quite  an  accomplishment.  In  my  opinion,  there  should  be  a 
third  type  of  award,  one  that  acknowledges  those  who  are  thoroughly  and  productively 
engaged  in  both  research  and  teaching  that  may  not  fit  the  direct  pure  research  criteria, 
where  the  other  types  of  research  are  valued  and/or  where  the  amount  of  research  may  be 
significant  but  less  than  those  so  highly  focused  on  only  research.  But  in  my  opinion, 
Boyer  (1990)  sets  the  stage  for  several  different  types  of  recognition.  Another  aspect  of 
these  awards,  and  I do  understand  the  financial  constraints,  is  that  there  may  be  more  than 
three  who  meet  the  criteria.  It  seems  to  me  that  once  a standard  is  set,  anyone  who 
achieves  the  criteria  should  receive  the  award.  I adamantly  believe  that  with  the  public 
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focus  on  teaching  and  the  private  partnerships  evolving  between  business  and  the 
university,  there  is  a way  to  fund  all  who  achieve  the  criteria. 

Boyer’s  (1990)  perspeetive  about  diseovery  [researeh]  is  ended  with  the  admonition,  that 
in  addition  to  seholars,  aoademie[s]  are  moral  teaehers.  By  their  own  style  of 
eonduct,  they  set  examples  for  the  next  generation  of  explorers.  It  thus  follows 
that  part  of  academic  duty  is  the  practice  of  eivility  in  seholarly  diseourse  - 
through  whieh  we  may,  by  example,  eneourage  the  kinds  of  attitudes  and 
behaviors  we  see  among  our  most  generous  eolleagues. . . objeetivity  and 
fairness. . . .The  university's  role. . .can  put  scholarship  in  its  proper  relationship  to 
other  forms  of  academic  duty . . . .It  must  be  in  balanee  with  other  obligations 
[duties]... also  giving  eonsideration  to  authorship,  priority,  and  eredit.  (pp.  184- 
185) 

Finally  when  addressing  one  aspeet  of  ehange  - the  intensely  eompetitive  atmosphere 
where  the  eentral  foree  in  that  ehange  is  the  faeulty,  they  remain  the  heart  and  mind  of  the 
university  and  their  eommitment  essential.  Where  creative  energy  and  institutional 
loyalty  exist,  faculty  members  are  willing  to  engage  in  experimentation;  where  it  does 
not,  they  do  not.  Interesting  to  note  is  that  Boyer  (1990)  says  the  elite  institutions  are  not 
the  ones  where  innovation  and  eommitment  are  evident,  but  instead  the  level  below  that 
top  rung  of  institutions  is  where  transformation  is  taking  plaee.  He  also  mentions  that  in 
more  stable  environments,  faculty  engagement  on  this  front  is  weaker,  requiring  new 
ways  to  make  faculty  feel  responsible  for  the  institution  and  students.  He  feels  shared 
govemanee  is  part  of  the  equation  so  faeulty  members  feel  like  stakeholders.  NIU  has 
this.  Leadership  is  eritieal,  and  leaders  must  advoeate  openly  for  their  own  faeulty  and 
public  understanding  of  them  and  their  work.  Institutional  redesign  must  eonsider  these 
and  other  dimensions  while  reclaiming  its  central  mission  as  soeiety's  agent  for  eultural 
transmission  and  eultural  ehange  by  working  through  thoughtful  partieipatory  transfer  of 
knowledge  and  excitement  from  one  generation  to  another,  always  putting  students  and 
their  needs  first.  Then  the  rest  will  fall  into  place,  including  the  tension  between  researeh 
and  teaehing.  This  will,  however,  require  altitudinal  ehange  and  the  emergenee  of  a new 
understanding  of  aeademie  duty. 

The  next  two  works  are  in  agreement  about  the  issues  diseussed  above,  the  history  and 
many  publie  perspeetives,  so  it  may  be  important  to  make  only  a few  of  their  points  not 
direetly  addressed  above.  For  example,  although  shared  govemanee  is  mentioned  above, 
O'Brien  (1998)  makes  the  point  that  the  dual  authority  of  faculty  and  administration,  or 
what  some  label  as  shared  governance,  is  one  of  the  reasons  that  most  world-elass 
universities  are  Ameriean  ones.  He  believes  that  we  are  strong  beeause  we  have  both 
strong  faeulties  and  strong  administration.  Also  in  agreement  with  the  above  authors,  but 
more  direetly,  he  is  impressed  with  the  persistence  of  the  moral  mission  of  the  modern 
university.  Another  important  aspeet  of  his  perspective  is  that  the  critical  discussions 
about  higher  edueation  are  mistaken  due  to  their  foeus  on  the  idea  of  higher  edueation 
rather  than  the  institution  of  higher  edueation  (p.  xviii).  His  identifieation  of  "half-truths" 
uses  the  definition  that  "what  is  claimed  is  essential  but  not  absolute"  (p.  5)  and  presents 
the  following  half-tmths: 
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• The  faculty  is  the  university.  He  argues  this  through  an  historical  perspective, 
ending  with  a thought  important  to  us:  "understanding  the  role  of  faculty  in  the 
research  university  is  the  prime  task  in  clarifying  the  nature  of  the  modem 
institution,  (p.  9) 

• Tenure  is  the  necessary  condition  of  academic  freedom  - not  the  same  as 
academic  freedom.  These  two  are  not  the  same  thing.  Tenure  is  a contract  for 
employment. 

• Universities  are  neutral  on  moral  value/universities  teach  moral  values  - these  two 
thoughts  do  not  add  up  to  a whole  truth.  There  must  be  "even  moral"  value 
embedded  in  the  collegiate  experience,  but  where? 

• The  liberal  arts  curriculum  aims  at  distribution/diversity.  Distribution  means  a 
well-rounded  education  - at  best  a gesture,  at  worst  a futile  gesture. 

• Diversity  on  both  moral  and  intellectual  grounds  is  proclaimed  by  institutions,  but 
what  does  it  really  mean?  It  is  often  a "code"  word  and  has  little  meaning  in 
either  sense. 

• Teaching  is  the  primary  task  of  higher  education.  Most  will  not  outright  deny  this 
claim.  He  feels  that  "the  emphasis  on  teaching  - and  the  much-publicized  conflict 
between  research  and  teaching  - is  downright  misleading. . . .The  proper  issue  is 
not  so  much  the  faculty  teaching  as  it  is  the  students  learning.  The  neglected  topic 
in  university  assessments  is  not  teaching;  it  is  [student]  learning  - with  or  without 
teachers  [faculty].”  (p.  12) 

The  others  include  the  problems  with  higher  education:  the  administration,  low-cost 
public  education  benefits  the  least  advantaged,  and  the  university  is  the  axial  institution 
of  modern  society.  Each  of  the  half-truths  is  more  deeply  explored. 

It  is  important  to  note  that  the  Illinois  public  has  issued  "A  Citizens'  Agenda  for  Illinois 
Higher  Education  - The  Illinois  Commitment:  Partnerships,  Opportunities,  and 
Excellence"  (IBHE,  1999;  Statewide  Progress,  2005).  Goal  2 "Higher  education  will  join 
elementary  and  secondary  education  to  improve  teaching  and  learning  at  all  levels" 
directs  us  to  build  further  strengths  in  the  educational  process  and  outcomes  (p.  4).  The 
progress  report  indicates  that  Illinois  graduates,  who  were  studied  in  a pilot  study  to 
develop  a model  for  collecting  and  analyzing  data  at  the  post  secondary  level,  have 
higher  levels  of  literacy,  and  are  better  prepared  for  the  baccalaureate  level.  Additionally, 
problem  solving  skills,  reading  comprehension,  and  writing  are  improving.  Important  to 
note  is  that  this  data  is  from  a very  small  sample,  thus  the  data  are  preliminary  at  best;  a 
larger  sample  could  contradict  these  results.  There  is  a direct  tie  to  our  college  initiative 
from  a "state"  directive. 

The  authors  above  describe  the  debate  and  issues  within  the  debate  regarding  higher 
education.  We  are  especially  interested  in  those  about  faculty  research,  the  Scholarship  of 
Teaching  as  acceptable  faculty  research,  and  how  to  make  research  on  teaching  an 
acceptable,  encouraged,  supported,  and  respected  area  of  research  for  promotion  and 
tenure  within  the  engineering  and  technology  context.  As  important  is  our  interest  in 
evolving  faculty  teaching  and  student  learning  to  higher  levels.  Therefore,  we  expect  the 
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outcomes  of  this  endeavor  to  impact  several  very  important  "fronts"  in  our  educational 
endeavors  in  CEET  at  NIU. 


Principles  to  Improve  Undergraduate  Education 

Chickering  and  Gamson,  through  The  American  Association  of  Higher  Education,  first 
published  the  “Seven  Principles  for  Good  Practice  in  Undergraduate  Education”  in  1987 
and  then  "Applying  the  Seven  Principles  for  Good  Practice  in  Undergraduate  Education" 
in  1991.  Hatfield  (1995)  explains  that  the  principles  were  not  new  then  and  that 
historically,  about  350  years  ago,  Comenius  called  for  active  learning:  "Eet  the  main 
object  of  our  didactic  be  to  seek  and  find  a method  of  instruction  by  which  teachers  may 
teach  less  so  that  learners  could  may  learn  more"  (p.  ix).  Another  important  statement  by 
Boyer  (1990)  was  “I  would  like  to  hear,  at  least  occasionally,  'Teach  or  Perish.'”  rather 
than  publish  or  perish  (as  cited  in  Hatfield,  p.  ix).  Since  then  the  Seven  Principles  have 
received  a great  deal  of  attention,  as  universities  have  begun  their  implementation  in  a 
myriad  of  ways.  The  principles  and  indicators  of  educational  effectiveness  for  each  are 

Principle  1:  Good  Practice  Encourages  Student-Faculty  Contact 
Erequent  student-faculty  contact  in  and  out  of  classes  is  the  most  important  factor  in 
student  motivation  and  involvement.  Eaculty  concern  helps  students  get  through  rough 
times  so  they  keep  on  working.  Knowing  a few  faculty  members  well  enhances  students' 
intellectual  commitment  and  encourages  them  to  think  about  their  own  values  and  future 
plans.  Indicators  of  effectiveness  are  (Hatfield,  1995,  p.  110) 

• Overall  student-faculty  ratio 

• Average  section  size 

• Percentage  of  sections  with  15  or  fewer  students  enrolled 

• Percentage  of  faculty  who  report  knowing  the  majority  of  their  students  by 

name 

• Percentage  of  students  involved  in  faculty  research 

• Students'  overall  frequency  of  out-of-class  contact  with  faculty 

• Student's  opportunity  for  in-class  discussion 

• Percentage  of  students  reporting  having  visited  faculty  during  established 
office  hours 

• Average  number  of  hours  faculty  spend  advising  each  week 

• Percent  of  students  reporting  after-class  conversations  on  academic  subjects 
with  faculty 

Principle  2:  Good  Practice  Encourages  Cooperation  among  Students  - Cooperative 
Learning  Communities 

Eeaming  is  enhanced  when  it  is  more  like  a team  effort  than  a solo  race.  Good  learning, 
like  good  work,  is  collaborative  and  social,  not  competitive  and  isolated.  Working  with 
others  often  increases  involvement  in  learning.  Sharing  one's  own  ideas  and  responding 
to  others'  reactions  improves  thinking  and  deepens  understanding.  Indicators  of 
effectiveness  are  (Hatfield,  1995,  p.  110) 

• Percentage  of  faculty  who  report  efforts  to  create  group  projects  or  learning 
communities  in  their  classes 
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• Percentage  of  courses  that  include  team  projects  or  similar  group  learning 
experiences 

• Percentage  of  students  participating  in  group  study 

• Percentage  of  students  reporting  frequent  out-of-class  discussions  with  fellow 
students  on  course  content 

• Use  of  non-competitive  grading  techniques  (competency-based,  criterion- 
referenced  vs.  use  of  the  normal  curve) 

Principle  3:  Good  Practice  Encourages  Active  Learning 

Learning  is  not  a spectator  sport.  Students  do  not  learn  much  just  sitting  in  classes 
listening  to  teachers,  memorizing  pre-packaged  assignments,  and  spitting  out  answers. 
They  must  talk  about  what  they  are  learning,  write  about  it,  relate  it  to  past  experiences, 
and  apply  it  to  their  daily  lives.  They  must  make  what  they  learn  part  of  themselves. 
Indicators  of  effectiveness  are  (Hatfield,  1995,  p.  110) 

• Percentage  of  faculty  using  teaching  techniques  that  involve  students'  active 
participation 

• Number  of  internships,  practical,  or  other  practice-oriented  courses  offered 
per  student 

• Number  of  independent  study  courses  offered  per  student 

• Percentage  of  graduating  seniors  engaging  in  at  least  one  internship, 
practicum,  independent  study,  or  similar  practice-oriented  course  (this 
assumes  high  quality  planning,  execution,  leadership,  and  requirements  for 
each,  especially  independent  studies) 

• Percentage  of  courses  requiring  students  to  engage  in  independent  research 
papers,  projects,  presentations,  or  similar  exercises 

• Percentage  of  courses  requiring  students  to  use  the  library  as  a research 
resource 

• Percentage  of  faculty  reporting  giving  students  credit  for  active  class 
participation 

• Percentage  of  students  reporting  using  the  library  as  a result  of  class 
assignment 

• Percentage  of  graduating  seniors  reporting  opportunities  for  field  projects  in 
courses  during  their  career 

Principle  4:  Good  Practice  Gives  Students  Prompt  Feedback 

Knowing  what  you  know  and  do  not  know  focuses  learning.  Students  need  appropriate 
feedback  on  performance  to  benefit  from  courses.  In  getting  started,  students  need  help  in 
assessing  existing  knowledge  and  competence.  In  classes,  students  need  frequent 
opportunities  to  perform  and  receive  suggestions  for  improvement.  At  various  points 
during  college  and  at  the  end,  students  need  chances  to  reflect  on  what  they  have  learned, 
what  they  still  need  to  know,  and  how  to  assess  themselves.  Indicators  of  effectiveness 
are  (Hatfield,  1995,  p.  Ill) 

• Average  number  of  graded  assignments  or  exercises  given  per  course 

• Average  number  of  graded  tests  and  quizzes  per  course 

• Average  "turnaround"  time  for  submission  of  final  course  grades 
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• Percentage  of  students  reporting  that  they  generally  received  graded 
assignments  back  from  professors  within  one  week 

• Percentage  of  students  reporting  that  professors  provided  frequent  and  specific 
oral  comments  on  performance 

• Percentage  of  students  reporting  that  the  professor  systematically  reviewed 
tests  or  assignments  in  class  after  papers  were  returned 

• Percentage  of  graduating  seniors  reporting  that  frequent  professor  feedback 
helped  improve  their  undergraduate  performance 

Principle  5:  Good  Practice  Emphasizes  Time  on  Task 

Time  plus  energy  equals  learning.  Efficient  time-management  skills  are  critical  for 
students  and  professionals  alike.  Allocating  realistic  amounts  of  time  means  effective 
learning  for  students  and  effective  teaching  for  faculty.  How  an  institution  defines  time 
expectations  for  students,  faculty,  administrators,  and  other  professional  staff  can 
establish  the  basis  for  high  performance  for  all.  Indicators  of  effective  are  (Hatfield, 

1995,  p.  112) 

• Average  student  course  load  taken  per  term 

• Percentage  of  courses  with  clear  attendance  policy 

• Average  amount  of  time  spent  studying  for  class  or  working  on  assignments 
per  week 

• Average  number  of  hours  per  week  spent  on  academic  assignments  as 
reported  by  graduating  seniors 

• Percentage  of  available  library  spaces  occupied  by  students  from  5-9  p.m. 
(which  has  changed  with  technological  advances  with  online  access  to 
information) 

Principle  6:  Good  Practice  Communicates  High  Expectations 

Expect  more  and  you  will  get  it.  High  expectations  are  important  for  everyone  - for  the 
poorly  prepared,  for  those  unwilling  to  exert  themselves,  and  for  the  bright  and 
motivated.  Expecting  students  to  perform  well  becomes  a self-fulfilling  prophecy  when 
teachers  and  institutions  hold  high  expectations  of  themselves  and  make  extra  effort. 
Indicators  of  effectiveness  are  (Hatfield,  1995,  (p.  113) 

• Average  ACT/SAT  of  entering  freshmen 

• Average  number  of  pages  of  reading  and  writing  assignments  in  humanities 
and  social  sciences  courses  (engineering  and  technology  courses  as  well) 

• Overall  distribution  of  grades  granted  each  term 

• Percentage  of  seniors  graduating  without  writing  a major  research  paper 
during  their  undergraduate  career 

• Percentage  of  students  reporting  not  being  significantly  challenged  by  class 
material  and  assignments 

Principle  7:  Good  Practice  Respects  Diverse  Talents  and  Ways  of  Learning 
There  are  many  roads  to  learning.  People  bring  different  talents  and  styles  of  learning  to 
college.  Brilliant  students  in  the  seminar  room  may  be  all  thumbs  in  the  lab  or  art  studio. 
Students  rich  in  hands-on  experience  may  not  do  so  well  with  theory.  Students  need  the 
opportunity  to  show  their  talents  and  learn  in  ways  that  work  for  them.  Then  they  can  be 
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pushed  to  learning  in  ways  that  do  not  come  so  easily.  Indicators  of  effectiveness  are 
(Hatfield,  1995,  p.  113) 

• Percentage  of  courses  requiring  students  to  speak  in  class  and  percentage 
requiring  students  to  view  visual  material  as  part  of  assignments 

• Percentage  of  students  reporting  that  they  were  encouraged  to  ask  questions  in 
class 

• Percentage  of  students  reporting  that  the  grading  and  evaluation  process  used 
by  the  professor  allowed  them  to  actually  demonstrate  what  they  knew 

• Percentage  of  students  reporting  that  performance  evaluations  permitted 
demonstrations  of  competencies  in  different  ways 

• Percentage  of  faculty  reporting  that  they  regularly  use  individualized  or 
alternative  forms  of  instruction  to  communicate  course  materials 

• Percentage  of  faculty  reporting  knowledge  about,  and  use  of,  formal 
differences  in  learning  styles  among  students  to  organize  their  courses 

In  consideration  of  Quality  Indicators  for  an  Educational  Environment,  Pascarella  and 
Terenzini  (1991)  identified  the  following  impacts  on  educational  environments  (as  cited 
in  Hatfield,  1995,  p.  108) 

• Eiving  on  campus  is  a critical  variable  to  college  impact,  so  student 
experience  other  forces  of  change  on  campus 

• Informal,  non-classroom  contacts  between  students  and  faculty  are 
responsible  for  increases  in  intellectuality,  maturity,  autonomy,  educational 
aspirations/attainments,  and  interpersonal  skills 

• Teaching  methods  that  stress  active  learning  produce  a wide  range  of 
intellectual  and  personal  effects 

• The  academic  major  influences  what  students  know  about  the  field 

• Relationships  with  peers,  while  not  as  influential  as  faculty  relationships  on 
intellectual  development,  are  important  to  other  areas  of  student  maturation 

Eearning  Communities 

This  section  will  present  concepts,  theory,  principles,  thoughts,  models,  and  practices  on 
learning  communities.  It  is  not  possible  to  consider  learning  communities  as  an  isolated 
concept;  therefore,  the  content  to  be  presented  will  be  messy.  It  will  flex  back  and  forth 
between  what  a learning  community  is  and  how  to  structure  and  develop  one,  while 
addressing  the  inherent  concepts,  theories,  principles,  definitions  and  more.  Eor  example, 
in  creating  such  a community,  one  has  to  understand  something  about  learning  in  general. 
A great  deal  of  the  literature  from  which  we  hope  to  be  informed  is  about  K-12  learning 
communities  of  teachers;  another  focus  of  the  broader  literature  is  about  student  learning 
communities  or  teacher-student/faculty-student  learning  communities.  All  the  authors 
present  a great  mixture  of  information  valuable  for  our  consideration.  Einally,  messy 
means  that  the  information  is  so  interrelated  that  it  is  difficult  to  present  linearly.  Eor 
example,  one  might  assume  that  I could  define  learning  communities  and  then  go  from 
there  - peeling  back  the  layers  of  the  topic,  or  begin  with  a foundation  definition  and 
build  from  there,  but  that  linear  thought  process  cannot  be  applied  to  an  integrated 
concept  or  entity  such  as  a learning  community.  Hopefully,  however,  one  reading  this 
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will  at  least  begin  to  create  one’s  own  foundational  framework  or  schema  from  which  to 
build  one’s  own  understanding  and  meaning;  expand  and  deepen  what  it  will  mean 
individually;  and  then  stimulate  engagement  with  the  group  to  create  the  faculty  learning 
community,  its  operational  context,  and  perhaps  beyond  that  faculty /student  LCs. 

In  this  section,  we  will  try  to  show  the  relationships  between  the  multiple  intelligences 
(MI),  knowledge(K),  learning(L)  and  its  processes,  and  change  through  learning 
organizations  (LO),  learning  communities  (LC),  learning  circles(lc),  communities  of 
practice  (CP),  and  communities  of  Interest  (Cl).  We  have  constructed  our  own  models 
and  processes  and  identified  our  own  program  content.  Let  us  begin  by  considering  first 
what  might  describe  a learning  organization.  Without  a doubt,  Senge's  (1990)  seminal 
work  on  learning  organizations.  The  Fifth  Discipline,  is  the  premiere  work  describing 
learning  organizations  and  inherently  the  learning  person.  His  work  with  other 
contributing  authors  on  The  Dance  of  Change  (1999)  and  Schools  that  Learn  (2000) 
informs  us  as  well.  These  works  will  be  used  to  establish  the  foundation  for  our  learning 
community  and  faculty  circles  of  team  members. 

Senge  (1990)  uses  the  engineering  discipline  to  describe  how  an  idea  moves  from 
invention  to  innovation.  He  says  that  particular  "component  technologies"  come  together 
where  isolated  developments  across  diverse  fields  of  research  come  together  in  an 
"ensemble  of  technologies  critical  to  each  other's  success”  (p.  5).  Until  this  ensemble 
forms,  the  idea  cannot  move  forward  and  achieve  its  potential  in  practice.  Along  that  line 
of  thought,  he  describes  five  component  technologies  that  converge  to  result  in 
innovation  in  learning  organizations.  Each  is  critical  to  the  others,  each  a dimension  in 
building  the  capacity  of  organizations  to  learn,  resulting  in  greater  capacity  to  achieve 
higher  potential.  The  components  are  systems  thinking,  personal  mastery,  mental  models, 
building  shared  models,  team  learning.  Each  of  these  dimensions  is  a discipline,  each  a 
body  of  theory  and  technique  to  be  learned  and  put  into  practice.  Each  is  a discipline  or 
developmental  path  for  acquiring  competencies  or  skills.  Neither  a person  nor  an 
organization  ever  truly  “‘arrives’  at  ‘the  place’  where  they  have  mastered  all  they  need  to 
know  or  learn,  for  that  is  forever  changing;  one  never  arrives  at  excellent,  for  that  bar  is 
always  being  raised;  but  an  individual  or  an  organization  can  practice  the  disciplines  of 
learning  and  become  better  or  worse”  (pp.  10-11).  When  the  component  technologies 
converge,  they  create  a new  wave  of  experimentation  and  advancement,  not  the  learning 
organization  but  rather  an  organization  that  is  always  “engaged  in  learning”  (p.  13).  True 
learning,  whether  in  an  individual  or  an  organization,  requires  "metanoia"  - a shift  of 
mind.  In  Greek,  it  meant,  "a  fundamental  shift  or  change,  or  more  literally,  transcendence 
("meta"— above  or  beyond,  as  in  "metaphysics")  of  mind  ("noia,"  from  the  root  "nous,"  of 
mind)”  (p.  13).  To  grasp  its  meaning,  that  of  metanoia,  is  to  grasp  the  deepest  meaning  of 
learning.  Teaming  involves  a shift  of  mind,  which  goes  far  beyond  taking  in  of 
information.  Eor  example,  if  one  reads  a book  on  bicycling,  one  cannot  really  say  that 
he/she  has  learned  how  to  ride  a bicycle.  Real  learning  means  that  "we  re-create 
ourselves.  Through  learning  we  become  able  to  do  something  we  never  were  able  to  do. 
Through  learning  we  re-perceive  the  world  and  our  relationship  to  it.  Through  learning 
we  extend  our  capacity  to  create,  to  be  part  of  the  generative  process  of  life"  (p.  14). 
Eurthermore,  this  is  not  only  the  meaning  for  learning  in  humans,  but  also  what  the  basic 
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meaning  of  learning  is  in  a learning  organization  - "an  organization  that  is  continually 
expanding  its  capacity  to  create  its  future. . .not  merely  to  survive. . .or  adapt  [to] 
learning. . .but  to  join  adaptive  learning  by  generative  learning,  learning  that  enhances  our 
capacity  to  create"  (p.  14).  This  definition  of  learning  truly  changed  my  life  and 
perspective  about  my  own  learning  and  my  teaching  practice.  The  sense  and 
understanding  from  Senge's  work,  that  learning  is  the  "re-creation  of  one's  self"  where 
one  "becomes  able  to  do  something  that  we  were  never  able  to  do. . .that  we  re-perceive 
the  world  and  our  relationship  to  it"  differently  has  great  meaning  and  insight  when  truly 
understood  (pp.  13-14).  This  became  the  goal  for  my  students  and  me.  Understanding 
something  of  each  of  the  component  technologies  builds  deeper  understanding  about 
learning. 

We  will  begin  and  end  with  the  Fifth  Discipline,  Systems  Thinking.  We  are  all  systems 
and  bound  by  the  interrelatedness  of  actions  on  each  other.  It  is  almost  impossible  to  be 
aware  of  the  impacts  our  actions  have  on  others;  for  in  time  and  space,  we  may  not  be 
together.  Therefore,  we  cannot  see  the  whole  pattern  of  change;  we  can  only  observe  the 
snapshots  of  isolated  parts  of  the  system,  which  makes  it  difficult  to  understand  the 
deepest  problems.  Systems  is  a conceptual  framework  where  one  can  use  a body  of 
knowledge  and  tools  to  make  patterns  across  systems  clearer  and  then  see  how  to  change 
the  patterns  that  have  negative  or  less  than  positive  effects.  Because  patterns  creep  up 
slowly,  we  are  not  aware  of  them,  and  thus,  they  can  do  great  damage.  A parable  to 
exemplify  that  is  the  "boiled  frog."  If  one  puts  a frog  into  a pan  of  lukewarm  water,  he 
will  sit  quietly  while  you  turn  up  the  heat.  The  heat  will  gradually  come  to  a boil  and 
literally  boil  the  frog  to  death.  That  is  the  way  of  slowly  developing  negative  or  harmful 
patterns.  They  gradually  do  great  harm  or  damage,  and  because  they  are  systems- 
oriented,  we  cannot  always  see  them  develop  unless  we  use  tools  and  techniques  to  study 
and  change  them  when  necessary  and  in  time.  Conversely,  we  react  to  crises  much  as  a 
frog  would.  When  one  puts  a frog  in  a boiling  pan  of  water,  he  immediately  hops  out.  He 
senses  the  danger  of  the  boiling  water.  Thus  we  often  spend  our  time  dealing  with  crises 
but  fail  to  realize  the  gradual  development  of  dangerous  patterns  can  also  cause  great 
damage.  But  as  part  of  systems,  it  is  difficult  to  discern  the  patterns  that  develop  when 
looking  through  the  lens  of  only  one  person  or  department.  We  must  consider  the  whole, 
the  system,  to  really  understand  events  across  time  and  space  and  their  interconnectivity, 
thus  interrelated  effects.  This  component  has  clear  implications  for  the  interconnectivity 
that  several  of  the  authors  above  present  between  teaching  and  research  - an  integrated 
system  where  one  cannot  exist  without  the  other.  The  first  discipline  in  Senge’s  (1990), 
Personal  Mastery,  may  be  interpreted  to  mean  "dominance  over  people  or  things”  (p.  8). 
However,  in  this  context  of  Senge's  learning  definition,  it  means  special  level  of 
proficiency.  People  with  a high  level  of  personal  mastery  are  able  to  consistently  realize 
the  results  that  matter  most  deeply  to  them.  In  effect,  they  approach  their  life  as  an  artist 
would  approach  a work  of  art,  committed  to  their  own  lifelong  learning.  It  is  a discipline 
of  continually  clarifying  and  deepening  our  personal  vision,  of  focusing  our  energies,  of 
developing  patience,  and  of  seeing  reality  objectively,  an  essential  cornerstone  of  the 
learning  organization  (and  the  learning  individual). 
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An  organization's  capacity  for  learning  can  be  no  greater  than  that  of  its  members  because 
few  organizations  encourage  the  growth  of  their  people,  which  results  in  vast  untapped 
resources:  people  enter  business  as  bright,  well-educated,  high-energy  people,  full  of 
energy  and  design  to  make  a difference.  By  the  time  they  are  30,  a few  are  on  the  “fast 
track”  and  the  rest  “put  in  their  time”  to  do  what  matters  to  them  on  the  weekend  lose 
commitment,  sense  of  mission,  and  excitement.  Those  with  personal  mastery  live  their 
lives  to  their  highest  aspirations,  in  other  words,  always  learning  through  self 
actualization.  Once  again,  personal  mastery  must  be,  or  should  be,  a core  competence  of 
any  professor.  Personal  mastery  is  all  about  the  continuous,  and  often  subliminal,  seeking 
of  learning  that  personifies  the  very  best  of  the  professoriate.  There  is  a quiet,  often  very 
personal,  need  and  drive  to  learn;  these  professors  are  never  “still.”  They  are  always 
seeking  new  projects,  research,  changing  their  approach  in  class,  moving  in  new 
directions.  Senge’s  (1990)  discipline  of  personal  mastery  describes  the  underpinning 
motivation  of  true  learners.  This  concept,  to  me,  also  clearly  separates  the  “learning” 
from  the  “learned.” 

Mental  Models,  the  second  discipline,  are  those  "deeply  ingrained  assumptions, 
generalizations,  even  pictures  or  images  that  influence  how  we  understand  the  world  and 
how  we  take  action"  (Senge,  1990,  p.  9).  Later,  this  will  be  connected  to  knowledge  as 
"frameworks,"  or  "structures"  and  assumptions  as  "paradigms"  within  which  we  operate 
to  learn  or  remain  the  same.  Senge  and  the  authors  below  all  mean  about  the  same  thing; 
our  mental  frameworks  are  embedded  assumptions  that  lead  us  to  take  or  not  take  action. 
To  take  action,  make  a change,  or  learn  would  require  adjusting  or  changing  internal 
assumptions  that  guide  one's  decisions  or  actions.  It  is  a type  of  "schema"  in  terms  of 
information  processing  or  framework  of  knowledge  and  experiences.  Mental  models 
guide  our  interactions  in  life;  therefore,  to  consider  new  directions  or  learn  new 
knowledge,  we  must  change  our  mental  model  by  turning  the  mirror  inward  and  engaging 
in  critical  reflection  (to  use  a term  later  used  herein).  This  means  that  we  challenge  our 
internal  pictures  of  the  world  by  surfacing  them,  scrutinizing  them,  and  then  carrying  on 
"learningful"  conversations  that  bring  a balance  to  inquiry  and  advocacy.  We  voice  our 
own  thinking  (later  referred  to  as  finding  our  own  inner  voices)  effectively,  possibly 
influencing  others  or  being  influenced  by  them.  We  hope  we  have  stimulated  this  in  our 
initial  professor  group  and  that  the  faculty  development  program’s  analysis  aspect  is 
based  upon  the  very  foundation  of  “critical  reflection,”  where  open  and  critical  self 
reflection  leads  us  and  motivates  us  to  change. 

The  third  discipline.  Building  Shared  Vision,  is  building  a picture  together  of  what  we 
can  be  or  what  we  would  seek  to  create  together,  the  capacity  to  hold  a shared  picture  of 
the  future  we  seek  to  create.  When  authentic  and  there  is  a "genuine  vision"  rather  than 
merely  a statement  of  vision,  people  learn  and  excel  because  they  can  visualize  where 
they  want  to  be  and  because  they  want  to.  Sometimes  visions,  good  visions,  get  stuck  in 
one  mind  and  are  not  shared.  Thus,  shared  visions  simply  means  "involving  people  in  the 
skills  of  unearthing  shared  'pictures  of  the  future'  that  foster  genuine  commitment  rather 
than  compliance"  (Senge,  1990,  p.  9).  At  best  we  hope  to  have  begun  some  of  the 
participating  faculty  members  on  a path  of  sharing  a different  vision  for  teaching  and 
learning  across  the  college.  We  realize  that  we  have  only  just  begun  to  present  the 
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opportunity,  develop  the  path,  and  try  to  motivate  some  of  our  faculty  members  to  enter 
the  pathway. 

Finally,  the  fourth  discipline.  Team  Learning,  involves  "dialogue,  the  capacity  of 
members  to  suspend  assumptions  and  enter  into  genuine  [authentic]  'thinking  together'" 
(Senge,  1990,  p.  10).  Once  again,  to  the  Greeks  dialogue  meant  "a  free-flowing  of 
meaning  through  a group,  allowing  the  group  to  discover  insights  not  attainable 
individually"  (p.  10).  Dialogue  is  quite  different  than  discussion  or  conversation;  it 
involves  "learning  how  to  recognize  the  patterns  of  interaction  in  teams  that  undermine 
learning.  Patterns  of  defensiveness  are  often  deeply  engrained  in  how  a team  operates 
[these  patterns  can  limit  a team’s  learning  and  insights].  If  unrecognized,  they  can 
undermine  learning.  If  recognized  and  surfaced  creatively,  they  can  actually  accelerate 
learning"  (p.  10).  Team  learning  is  critical,  where  teams,  not  individuals,  are  the  learning 
unit.  If  teams  learn,  organizations  learn;  if  teams  cannot  learn,  then  organizations  cannot 
learn.  This  concept  sets  the  stage  for  learning  communities  but  is  not  something  that 
happens  immediately.  We  believe  that  our  small  group  engaged  in  a great  deal  of 
individual  learning,  a lot  of  conversation,  and  some  dialogue.  There  was  some  team 
learning,  but  we  are  at  the  very  beginning  level  of  team  learning.  True  team  learning  is 
something  that  will  expand  and  deepen  with  more  opportunities  to  work  together,  and 
let’s  face  it,  we  had  a lot  to  accomplish  individually  in  the  “self-critical  reflection”  aspect 
of  the  program  before  we  could  move  to  “critical  reflection  as  a group.” 

It  is  important  to  note,  however,  that  our  particular  group  did  not  seem  intimidated  by 
examining  themselves,  their  philosophies,  and  professional  practices,  and  they  seemed  to 
openly  engage  in  critical  reflection.  They  made  major  changes  to  their  courses,  syllabi, 
and  tests,  adding  performance  assessments  and  more.  They  openly  documented  their 
strengths  and  weaknesses.  They  did  not  shy  away  from,  nor  withdraw  from,  “critical” 
self-reflection.  I believe  this  is  because  they  all  genuinely  care  about  their  teaching  and 
student  learning.  But  to  broaden  and  deepen  team  learning,  where  the  sum  of  the 
individual  learning  is  truly  greater  than  the  number  on  the  team,  is  a step  we  hope  to  take 
next.  Let  us  wrap  these  thoughts  by  revisiting  the  fifth  discipline.  Systems  Thinking,  as  it 
is  the  discipline  that  integrates  the  others:  "fusing  them  into  a coherent  body  of  theory 
and  practice"  (Senge,  1990,  p.  13).  Mental  models  lead  to  openness  and  ability  to 
discover  weaknesses.  Team  learning  engages  groups  in  looking  for  the  larger  picture,  and 
personal  mastery  engages  each  of  us  in  the  personal  motivation  to  continue  learning  and 
growing.  Otherwise,  we  would  be  reactive  (“others  create  my  problems”)  and  where  we 
would  not  "own"  any  part  in  creating  our  own  problems.  The  integration  of  the  whole 
enlightens  us  and  we  begin  to  re-perceive  ourselves  and  our  relationship  to  the  world. 
Learning  is  a "shift  of  mind  - from  seeing  ourselves  as  separate  from  the  world  to 
connected  to  the  world,  from  seeing  problems  caused  by  someone  else  or  something  'out 
there'  to  seeing  how  our  own  actions  create  the  problems  we  experience”  (p.  13).  A 
learning  person  and  a learning  organization  are  where  that  individual  and  those  people 
continually  discover  how  they  create  their  own  reality.  They  engage  in  “creative  tension” 
by  holding  a vision  and  concurrently  telling  the  truth  about  the  current  reality  relative  to 
that  vision.  Creative  tension  can  be  used  to  “lead  people  to  see  more  aspects  of  reality  as 
something  they  can  individually  and/or  collectively  influence.... and,  how  they  can 
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change  it"  (King,  1986;  Senge,  1990,  p.  357).  A learning  organization  or  a learning 
person  or  team  occurs  when  people  are  continually  discovering  how  they  can  create  their 
own  reality  or  change  it. 

In  The  Dance  of  Change  (1999),  Senge,  Kleiner,  Roberts,  Ross,  Roth,  and  Smith 
establish  that  change  must  be  led  from  top  down  because  nothing  can  happen  without  the 
CEO  or  Dean  on  board  or  buying-in.  However,  they  go  on  to  acknowledge  that  little  or 
no  significant  change  will  occur  if  change  is  ONLY  driven  from  the  top.  There  must  be 
genuine  commitment  and  learning  capability  throughout  the  organization  - in  our  case, 
the  college.  Thus,  we  began  with  a small  but  committed  group,  at  least  at  the  level  of 
being  willing  to  complete  all  aspects  of  the  faculty  development  program  and  to  continue 
with  the  experimental  research  in  the  classrooms,  submitting  research  publications  and 
participating  in  presentations.  There  seems  to  be  a commitment  by  the  professor  group,  as 
well,  to  revise  their  other  classes  at  some  point  in  time,  although  that  level  of 
commitment  remains  to  be  examined  at  a later  point.  Some  professors  seem  to  be 
interested  in  becoming  the  faculty  development  leaders  to  sustain  the  program;  therefore, 
we  have  leadership  at  the  top  and  evidence  of  rather  strong  faculty  commitment.  (Long 
term  commitment  cannot  be  addressed  at  this  point.)  When  considering  leadership, 
obviously  we  are  not  talking  about  the  Dean  as  the  only  primary  since  this  program  was 
developed  and  led  by  a professor  and  NIU  professional  staff  member.  The  Dean 
empowered  them  by  acknowledging  the  need,  leading  the  challenge  for  change,  and 
supporting  the  program  and  development.  This  exemplifies  true  empowerment  and 
transforming  leadership  (Burns,  1978)  and  superleadership  (Manz  & Sims,  1989;  Pearce 
et  al,  1994;  Yukl,  1989),  where  the  leader  raises  him/herself  and  others  to  higher  levels  of 
aspiration  and  develops  others  into  leaders,  which  leads  to  community  building  and 
leaders  known  as  “network  leaders”  or  “community  builders”  (Senge  et  ah,  p.  17). 

In  moving  to  acknowledge  Boyer’s  (1990)  Scholarship  of  Teaching,  one  must  first  build 
a community,  leadership,  a foundation  for  networking,  knowledge,  and  support  for 
learning  and  action  to  result.  The  challenges  along  the  way  are  simply  stated  by  Senge  et 
al.  (1999):  “We  don’t  have  time  for  this  stuff!”  “We  have  no  help!”  “This  stuff  isn’t 
relevant!”  “They’re  not  walking  the  talk!”  ”My  situation  (my  students,  my  course)  is 
different!”  “We  keep  reinventing  the  wheel.”  “Where  are  we  going?”  “What  are  we  here 
for?”  In  our  initiative,  we  tried  to  address  all  of  this  up  front,  and  I believe  we  did  that 
fairly  successfully.  The  Dean  and  program  leaders  backed  up  all  theory  with  research  or 
nationally  recognized  practice.  Each  program  goal  was  justified  by  national  standards, 
accreditation  criteria,  or  communities  of  practice  demands.  The  initiative  goals  were 
clearly  established.  Professors  were  provided  time  and  stipends  to  participate;  they  were 
fully  supported;  and  the  Dean  was  active  and  visible.  Professors  were  given  a complete 
“tool  box”  of  materials  - copies  of  all  major  literature,  the  draft  of  this  review,  nationally 
recognized  sources  to  guide  practice  and  process,  national  standards,  and  more.  They 
were  led  through  the  entire  development  process.  In  our  case  this  meant  they  actually 
redesigned  and  developed  their  courses,  selected  new  teaching  strategies,  designed  and 
developed  new  tests  and  performance  assessments,  and  much  more  while  together  with 
the  program  leaders  throughout  18  days.  They  were  expected  to  have  to  do  very  little 
outside  of  “class”  time.  They  were  paid  a stipend,  one  half  at  the  end  of  professional 
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development  and  one  half  upon  completing  their  research  manuscript.  Some  program 
days  were  during  the  normal  semester,  and  some  were  during  the  summer.  We  tried  to 
accommodate  every  challenge  and  potential  resistance  in  the  program  design,  at  least  to 
get  the  initiative  started  towards  the  Scholarship  of  Teaching  in  the  college. 

In  The  Dance  of  Change,  Senge  et  al.  (1999)  make  a point  that  is  truly  the  foundation  for 
any  “profound”  change.  Is  the  change  to  be  “authority”  or  “learning”  driven?  (p.  41). 
Authority  driven  change  is  easier  to  organize  and  can  be  more  easily  effective  short  term. 
However,  deep  and  potentially  sustainable  change,  where  change  continues  to  occur,  is 
driven  by  learning,  where  there  are 

repeated  opportunities  for  small  actions  that  individuals  could  design,  initiate,  and 
implement  themselves. . .first  on  a small  scale  and  then  with  increasingly  larger 
numbers  of  people  and  activities,  [where]  people  articulate  the  goals  they  would 
like  to  achieve,  experiment  with  new  projects  and  initiatives,  learn  from  their 
successes  and  mistakes,  and  talk  with  each  other,  candidly  and  openly,  about 
results. . .draw[ing]  in  new  people  who  share  similar  values  and  aspirations”  or 
who  learn  that  they  do  (p.  41). 

This  is  where  we  want  to  be  with  our  initiative  and  is  why  we  began  with  learning  as  the 
foundation  and  operational  context  - the  ultimate  meaning  of  learning  is  change.  Senge’s 
group  identified  the  qualities  in  which  the  most  important  change  initiatives  seem  to  be 
grounded: 

• They  are  connected  with  real  work  goals  and  processes; 

• They  are  connected  with  improving  performance; 

• They  involve  people  who  have  the  power  to  take  action  regarding  these  goals; 

• They  seek  to  balance  action  and  reflection,  connecting  inquiry  and 
experimentation ; 

• They  afford  people  an  increased  amount  of  “white  space”  opportunities  for 
people  to  think  and  reflect  without  pressure  to  make  decisions; 

• They  are  intended  to  increase  people’s  capacity,  individually  and  collectively; 
and 

• They  focus  on  learning  about  learning  in  settings  that  matter,  (p.  43) 

This  is  how  we  tried  to  develop  our  context  for  this  change  initiative,  although  we  did 
make  many  instructional,  teaching  and  learning  decisions.  Clearly,  the  participating 
faculty  were  supported  to  make  changes  related  to  goals,  processes,  and  performance  and 
were  supported  to  inquire  through  experimentation,  etc.  The  whole  endeavor  was  about 
learning;  however,  not  just  student  learning  - but  about  their  responsibility  to  be 
“learning”  faculty  members  if  they  were  to  truly  model  and  be  capable  of  leading 
students  to  learn.  This  requires  “aspiration... reflective  conversation... understanding 
complexity... capabilities  all  learners  must  have”  (Senge  et  al.,  1999,  p.  45).  People  are 
passionate  about  producing  results,  but  they  must  see  that  it  matters,  that  their  colleagues 
take  it  seriously,  and  that  because  it  works  - it  gets  results.  We  tried  to  design  our 
program  to  reflect  these  operational  premises.  We  recognized  that  commitment  initially 
was  only  possible  with  a selected  small  group  of  professors;  however,  our  goal  is  to  grow 
steadily  by  providing  useful  tools  and  processes  in  a supportive  context.  We  have  tried  to 
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discern  our  limitations,  understand  the  current  reality  and  practice,  acknowledge  existing 
mental  models,  and  situate  professors  to  prove  what  works  to  themselves  and  their 
colleagues  through  experimentation.  We  realize  that  although  we  engaged  them  in  18 
eight-hour  days  of  development,  this  is  not  realistic  for  the  long  term.  However,  now  that 
the  program  has  been  piloted,  we  know  where  to  revise  and  modify  it  in  content  and 
process  - very  possibly  with  a shorter  time  requirement.  There  are  challenges  that  we  will 
need  to  address  and  consider,  and  we  need  to  remember  that  we  are  initiating  a culture 
change  in  teaching  and  learning  and  also  in  what  is  acceptable  and  encouraged  as 
scholarship  and  research.  For  an  engineering  and  technology  college,  this  is  a major 
culture  shift  to  include  and  develop  respect  for  the  scholarship  of  teaching,  even  with  a 
long  productive  history  of  research  on  K-12  education. 

Another  aspect  of  that  culture  change  is  collaboration  - learning  communities.  We  are 
seeking  to  engage  faculty  members  in  individual  and  collaborative  scholarship  on 
teaching  and  learning.  Although  many  faculty  members  collaborate  on  research  and 
private  contracts,  the  culture  does  not  include  collaboration  on  teaching;  therefore,  this  is 
a new  shift  as  well.  And,  regarding  ongoing  assessment  and  evaluation,  we  hope  to 
modify  our  faculty  evaluation,  promotion  and  tenure  criteria  to  reflect  the  culture  and 
scholarship  changes.  (That  is  part  of  the  phases  to  follow  the  program  and  classroom 
pilots’  analyses  and  results.)  This  will  be  reflected  in  revised  bylaws  and  faculty 
evaluation  criteria.  Since  we  are  a “shared  governance”  institution,  the  bylaws  require  a 
2/3  majority  vote;  however,  we  do  not  expect  difficulty  since  we  are  striving  to  broaden 
opportunity  rather  than  restrict  or  increase  the  level  of  productivity. 

In  Schools  that  Learn  (2000),  Senge,  Cambron-McCabe,  Lucas,  Smith,  Dutton,  and 
Kleiner  clearly  establish  that  a school,  whether  K-12  or  college  level,  is  a “fulcrum  point 
for  educational  and  societal  change”  (p.  6).  They  make  a very  good  point  that  classrooms 
can  only  improve  if  the  schools  and  communities  improve  and  that  learning  schools  are 
not  so  much  a separate  place  as  a meeting  ground  for  learning  - dedicated  to  the  idea  that 
those  involved,  individually  and  collectively,  are  continuously  expanding  their 
capabilities  through  learning  as  described  in  The  Fifth  Discipline  within  the  context  of  a 
community.  In  a learning  community,  everyone  is  involved  in  learning  and  supporting 
learning.  The  illustration  below  shows  that  the  domain  of  action  requires  guiding  ideas, 
innovations,  theory,  methods,  and  tools  to  make  change  happen  in  a way  that  it  can 
endure.  In  other  words,  awareness  changes  sensibilities,  attitudes  and  beliefs,  skills  and 
capabilities,  resulting  in  a deeper  cycle  of  learning. 
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Figure  A.6.1:  Domain  of  Action 


Innovations  in  Infrastructure  Theory,  Methods,  and  Tools 

(Senge,  et  al,  2000,  p.  26) 

Senge  et  al.  (2000)  define  learning  as  connections  in  which  one  of  the  most  challenging 
aspects  is  to  get  teachers  to  understand  that  there  are  others  in  the  classroom  with  them 
and  that  they  are  teaching  students  and  subject  matter.  Knowledge  is  separated  from  the 
knower,  where  teachers  deposit  “tokens  of  codified  knowledge,  discrete  pieces  of 
information,  into  students’  heads”  (p.  21).  However,  fields  of  knowledge  do  not  exist 
separately  from  each  other,  nor  are  they  separate  from  those  who  study  them. 

“Knowledge  and  learning  - the  processes  by  which  people  create  knowledge  - are  living 
systems  made  up  of  often-invisible  networks  and  interrelationships  - a complex  living 
system”  (p.  22).  This  greatly  affects  the  ability  for  individuals  or  groups  to  learn. 

All  learners  construct  knowledge  from  an  inner  scaffolding  of  their  individual  and 
social  experiences,  emotions,  will,  aptitudes,  beliefs,  values,  self-awareness, 
purpose,  and  more.  Thus,  one’s  learning  is  determined  by  understanding,  who  you 
are,  and  what  you  already  know  - the  schema  or  framework  that  one  connects 
new  learning  to.  Learning  is  affected  by  what  is  covered,  how  it  is  taught  or 
learned,  and  who  teaches  it.  These  connections  or  disconnections  can  strengthen 
or  weaken  learning,  (pp.  167-168) 

Teaching  and  learning  are  driven  by  vision,  according  to  Senge  et  al.  (2000),  making  it 
more  possible  for  those  responsible  for  the  educational  process  to  understand  what  they 
are  trying  to  accomplish.  So  in  considering  an  educational  vision,  are  we  trying  to 
“socialize  young  people  so  they  fit  into  society. . .train  a workforce. . .introduce  young 
people  to  greater  possibilities  for  their  lives?”  (p.  168).  The  most  profound  answer  is 
“helping  young  people  learn  how  to  create  the  lives  they  truly  want  to  create”  (p.  169). 

To  accomplish  this  creative  process,  a necessary  key  is  “structural  tension”  - contrast. 
Tension  creates  a need  for  resolution;  the  “contrast  is  a state  between  our  desired  state  or 
goals,  aspirations,  desires  and  our  current  reality.  In  resolving  the  tension  by  taking 
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actions,  we  achieve  our  goals  or  future  state”  (pp.  167-168).  To  describe  the  context 
within  which  we  are  working,  it  might  help  to  consider  Senge  et  al.’s  perspective  of 
disciplines  as  unnatural;  in  other  words  we  must  define  the  desired  result  and  define 
reality  objectively  by  removing  ourselves  from  our  assumptions,  theories,  and  concepts. 
That  requires  discipline,  an  unnatural  state,  to  achieve  the  objectivity,  ultimately  learning 
from  mistakes.  Discipline  is  what  builds  the  capacity  to  maintain  structural  tension  - 
creative  tension  that  can  result  in  change.  When  one  has  great  capacity  to  establish  and 
maintain  structural  tension,  there  can  be  great  personal  mastery;  otherwise,  whenever 
someone  becomes  uncomfortable,  he/she  will  give  up.  Another  requirement  is  to  ask 
students  what  they  want,  but  differently  than  how  we  typically  ask  the  question. 

Typically,  when  students  are  asked  that  question,  it  is  within  a realm  of  established 
possibilities  - out  of  what  is  made  available.  This  is  compliance  oriented.  The  difference 
between  that  and  a better  way  to  ask  the  question  is  to  ask  in  a way  that  allows  responses 
not  on  the  menu  of  acceptable  possibilities.  That  way,  we  are  not  suggesting  that  they 
limit  their  aspirations  to  something  reasonable  but  rather  that  they  become  “creative”  or 
“self-generative,”  realizing  choices  not  already  established  or  considered  what  they 
“should”  want  (p.  170).  If  we  teach  them  to  love  being  creative  and  generative,  then 
students  will  be  motivated  to  learn  what  they  need  to  learn  to  accomplish  what  they  want 
to  create  or  what  they  need  to  change  to  accomplish  their  goals.  Furthermore,  we  need  to 
steer  away  from  what  Senge  and  his  colleagues  call  the  “perfectionist  virus”  or  the 
“canned  answer”:  in  other  words,  learning  that  results  in  the  expected  answer  or  evidence 
of  learning  process.  This  can  result  in  students  feeling  they  are  not  learning  anything  of 
value  because  they  realize  the  right  answer  is  to  mimic  the  ideal  end  product  expected 
“rather  than  understanding  the  requisite  knowledge  and  strengthening  their  learning 
process”  (p.  184).  When  students  feel  they  are  not  learning  anything  they  can  really  use, 
their  motivation  decreases  and  they  lose  interest,  often  becoming  low  performers  even 
though  bright  and  capable.  If  they  feel  they  cannot  “mimic  the  answer  effectively  that  it  is 
their  fault. . ..The  safety  net  for  taking  risks  is  removed;  they  will  no  longer  be  permitted 
to  fail. . . .They  become  anxious,  unsatisfied  perfectionists,  guilty  procrastinators,  or  both. 
Self  blame  can  spread  from  student  to  student. . .ultimately  shutting  down  learning  and 
flow  of  information”  (p.  185). 

Senge  et  al.  (2000)  support  “assessment  as  learning”  (p.  205)  (also  discussed  further 
later).  Considering  formal  (explicit,  codified  facts),  applicable  (transferring  knowledge 
into  action),  and  longitudinal  knowledge  (capacity  to  act  effectively  over  time,  using 
knowledge  more  effectively  and  innovatively),  good  formal  tests  do  not  assess  learning 
authentically  nor  do  they  reveal  the  progress  students  are  making  or  motivate  students  to 
reflect  upon  what  they  know  and  what  they  still  need  to  know  or  learn.  Instead  “we  need 
assessments  that  are  designed  for  learning,  not  assessments  that  are  used  for  blaming, 
ranking,  and  certifying. . .requir[ing]  a shift  of  attitude  about  testing  and  learning.  We 
need  evaluation  that  is  more  informed”  (p.  205).  The  qualities  they  suggest  for 
assessment  for  learning  are  timeliness,  or  immediate  and  ongoing  feedback;  the 
timeliness  of  the  feedback  directly  impacts  its  meaningfulness  and  value  to  the  student. 
Honesty  is  another  quality  that  is  critical,  as  it  causes  “disequilibrium”  or  “cognitive 
dissonance,”  making  students  face  the  feedback  data  and  what  is  means,  possibly  the 
need  for  change.  Reflection  may  be  one  of  the  most  important  qualities  where  the  learners 
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realize  their  “spiral”  of  learning  or  progress.  Their  reflection  reveals  that  they  have  made 
progress  through  the  plan,  take  action/experiment,  assess/gather  evidence,  study /reflect, 
and  modify  actions  based  on  new  knowledge  spiraling  cycles.  We  do  not  build  in  enough 
reflective  time  so  students  can  see  where  they  are  and  what  they  still  have  to  learn  - self- 
evaluation  - or  how  to  plan  and  organize  their  own  learning  based  upon  what  they 
learned  from  the  feedback.  The  intellectual  behaviors  we  want  to  instill  are 

• to  persist  in  learning,  using  alternative  strategies  if  solutions  are  not  immediately 
apparent;  to  decrease  impulsivity  - 

• to  think  through  the  problem  and  use  the  problem  solving  process  and  alternative 
strategies; 

• to  listen  to  others  and  to  understand  others’  points  of  view  - some  feel  this  is  the 
most  intelligent  behavior; 

• to  think  with  flexibility  - once  again  considering  alternative  points  of  view  rather 
than  their  own  way  as  the  “only  way”  to  solve  a problem; 

• to  employ  metacognition  - being  aware  of  their  own  thinking  processes. . .They 
can  describe  their  own  processes  and  are  able  to  evaluate  themselves;  striving  for 
accuracy  and  precision,  making  sure  that  their  work  is  complete  and  accurate, 
reflecting  on  it  before  turning  it  in  to  quickly; 

• to  have  the  ability  to  question  and  pose  problems,  risking  others’  opinions; 

• to  have  the  ability  to  use  past  knowledge  and  experiences  in  different  contexts, 
abstracting  learning  from  one  meaning  or  experience  to  another; 

• to  have  the  capacity  for  ingenuity,  originality,  insightfulness,  creativity  - 
capability  to  be  generative; 

• to  have  the  ability  to  express  themselves  and  their  thoughts  with  precise  language; 

• to  have  the  ability  to  gather  data  using  all  the  senses;  demonstration  of  a sense  of 
humor;  inquisitiveness; 

• to  have  cooperative  intelligence  and  social  intelligence  - learning  and  working 
with  others  successfully,  (pp.  197-205) 

One  of  the  most  illuminating  sections  in  Schools  that  Learn  (2000)  is  about  pedagogy  and 
literacy.  In  thinking  about  pedagogy,  one  might  want  to  consider  what  literacy  has  meant 
- “educated,  learned”  (p.  210).  Or  one  might  want  to  realize  that  in  teaching  students  to 
become  literate,  literacy  has  become  “literacies,”  no  longer  meaning  just  reading,  writing, 
speaking,  or  fundamental  mathematics.  Today,  students  must  become  computer  literate, 
technologically  literate,  and  scientifically  literate,  as  well  as  have  cultural,  environmental, 
visual,  financial,  musical,  etc.  literacy.  Thus,  when  determining  appropriate  pedagogy, 
Senge  et  al.  (2000)  are  in  agreement  with  Gardner  when  he  describes  multiple 
intelligences  and  that  we  should  strive  to  teach  for  all  intelligences.  Each  literacy  is  a 
“form  of  power  . . .and  a kind  of  leverage”  (p.  210). 

Senge  et  al.  (2000)  discuss  pedagogy  somewhat  differently.  If  “literacy  is  a form  of 
power,”  then  teaching  literacy  has  inherent  power  related  issues.  They  present  an 
example  in  which  reading  and  writing  literacy  gives  individuals  power  over  “the 
symbolic  world  of  reading  and  writing,”  which  in  turn  gives  them  the  power  to  “affect  or 
transform  the  world”  (p.  211).  They  present  three  pedagogical  approaches: 
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• Transmission  Pedagogy  - takes  power  away  from  the  learner. . .and. . .the  teacher; 

• Generative  Pedagogy  - grants  teachers  and  learners  the  power  to  relate  to  the 
subject  matter  and  build  on  their  knowledge; 

• Transformative  Pedagogy  - provides  learners  with  a functional  literacy  and 
provides  teachers  and  learners  with  a social  literacy,  or  systems  literacy,  which 
gives  them  the  power  to  create  their  desired  future,  (p.  211) 

When  considering  what  “power”  means,  we  sometimes  forget  that  it  can  be  positive  and 
negative.  Its  Latin  base  means  “be  able”;  in  French,  it  means  “ability  to  do  things.” 
Therefore,  power  does  not  always  mean  or  relate  to  control  and  authority.  Often  people 
draw  on  their  own  power  to  increase  their  capabilities.  If  we  move  to  “empower,”  there 
might  be  an  underlying  assumption  that  the  receivers  have  no  power  other  than  that  given 
to  them  and  that  “the  internal  powers  that  drive  us  as  human  beings  are  valid  only  when 
granted  by  external  agents  who  possess  knowledge,  authority,  or  control”  (p.  211).  These 
authors  strive  to  create  awareness  that  “power  from  the  outside  (an  individual,  group,  or 
organization),  especially  when  unseen  and  unacknowledged,  often  disconnects  people 
from  their  [own]  potential  power  from  within”  (p.  211).  They  quote  Freire  on  critical 
pedagogy.  Schools  are  political  sites  and  no  content  or  process  of  teaching  and  learning  is 
politically  neutral.  “Literacy  is  best  understood  as  a myriad  of  [communication]  forms 
and  cultural  competencies  that  construct  and  make  available  the  various  relations  and 
experiences  that  exist  between  learners  and  the  world”  (p.  211). 

I have  often  questioned  others  on  “what”  they  teach  as  well  as  “how”  they  teach.  For 
example,  in  history,  students  often  learn  about  or  memorize  a chronology  of  events,  e.g. 
wars.  They  come  away  knowing  about  events  and  timelines.  However,  that  is  not  the 
“content”  or  “what”  they  should  be  learning  about.  For  me,  history  is  all  about  cause  and 
effect  - what  caused  the  wars;  what  impact  the  wars  had  on  the  people,  economy, 
development,  what  consequences  or  outcomes  resulted?  Onward,  what  could  have 
prevented  the  wars;  what  were  the  real  issues  that  led  to  the  war?  These  patterns  require 
thinking  at  a more  critical  and  analytical  level;  whereas  learning  about  events  and 
timelines  do  not.  Events  and  timelines  do  not  lead  to  literacy,  but  consideration  of  critical 
question  and  examining  underlying  causes  and  effects  certainly  do.  In  science, 
engineering  and  technology,  the  same  is  true.  Systems  thinking  leads  us  to  use  methods 
such  as  causal  loops,  which  reveal  patterns  that  repeat  themselves  across  contexts  or 
stock  and  flow  diagramming,  simulations,  modeling  software,  concept  mapping,  etc. 
Memorizing  the  table  of  elements  does  not  lead  one  to  understand  chemistry,  the  results 
of  combining  elements  given  particular  conditions,  constraints,  tools,  etc.  Another 
example,  we  can  teach  students  how  to  use  formulas  in  algebra  or  physics,  but  do  they 
understand  the  individual  factors  and  the  relationships  between  the  factors  in  the 
formulas,  so  when  they  are  presented  with  a problem  they  can  create  a formula 
appropriate  for  that  particular  problem.  This  group  also  advocates  strongly  for  “learner- 
centered”  learning  and  “bridging  subject  boundaries”  or  integrated  interdisciplinary 
curricula.  This  is  discussed  in  more  detail  later,  but  learner-centered  teaching  and 
learning  engages  the  students  actively  and  more  authentically  in  learning  and  assessment 
that  is  as  real  world  related  as  possible,  where  the  burden  of  learning  or  discovery  or 
solution  is  on  the  students  and  not  where  the  teacher  “imparts”  knowledge.  It  is  usually 
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problem  based;  however,  it  is  important  to  note  that  not  all  the  answers  are  known  to  the 
problems.  Therefore,  different  students  or  groups  could  have  different  results  based  upon 
their  strategies  of  inquiry  and  development,  choice  of  alternatives,  learning  styles, 
problem  solving  skills,  etc. 

To  affect  change  in  learning,  the  assumptions  about  learning  must  be  challenged:  1) 
schools  fix  deficiencies,  2)  learning  takes  place  in  the  head  rather  than  the  body  as  a 
whole,  3)  everyone  learns  in  the  same  way,  4)  learning  takes  place  in  the  classroom  rather 
than  the  real  world,  and,  5)  there  are  smart  kids  and  dumb  kids.  Also  we  must  challenge 
the  assumptions  about  schools:  1)  schools  are  run  by  specialists  who  are  in  control,  2) 
knowledge  is  fragmented,  3)  schools  reveal  the  “truth,”  and  4)  learning  is  individualistic 
and  competition  accelerates  learning.  If  we  believe  the  above  assumptions,  then  learning 
is  presented  as  a machine-like  model  and  process  (Senge  et  ah,  2000,  pp.  35-50). 
However,  if  the  model  for  learning  and  schools  becomes  dynamic  - a living  system 
where  learning  focuses  on  subjects  or  content  as  if  alive,  then  the  educational  processes 
become  “learner  centered  rather  than  teacher  centered. . .encourage  variety,  not 
homogeneity,  embracing  multiple  intelligences  and  diverse  learning  styles;  and 
understanding  a world  of  interdependency  and  change  rather  than  memorizing  facts  and 
striving  for  right  answers”  (p.  55).  This  means  that  educators  are  “constantly  exploring 
the  theories-in-use  of  all  involved  educational  process;  and  reintegrating  education  within 
webs  of  social  relationships  that  link  friends,  families,  and  communities”  (p.  55). 

Ultimately,  learning  takes  place  in  a community,  which  ranges  from  its  Indo-European 
root  “everyone”  and  “exchange”  to  become  “shared  by  all.”  It  evolved  in  Latin  to  mean  a 
“source,”  in  French  to  mean  “to  make  available  to  everyone.”  What  is  important  is  that 
“community”  really  does  not  mean  a “place”  defined  by  boundaries  but  by  the  “sharing 
of  life”  (Senge  et  ah,  2000,  p.  461).  Senge’s  (2000)  group  views  community  to  mean 
a group  of  people  or  relationships  within  an  organization  -as  a ‘community  of 
practice’  or  ‘learning  community’  within  a school. . .a  community  of  people  is  a 
place  rooted  in  the  biosphere,  rife  with  activity,  mutual  respect,  and  the 
recognition  that  everyone  in  that  place  is  responsible  for  and  accountable  to  one 
another,  because  the  lives  of  all  are  interdependent,  (p.  461) 

Therefore,  a community  that  learns  requires  three  different  kinds  of  activities  for  a 
learning  approach  to  develop:  defining  their  identity;  building  connections;  and 
sustainability  - a long-term  perspective  and  an  understanding  of  interdependence  (Senge 
et  ah,  2000,  pp.  461-465).  Community  has  always  been  critical  if  learning  is  to  occur  very 
well,  as  learning  is  a social  process  that  helps  individuals  realize  their  learning  potential 
within  a learning  community  that  is  supportive,  stimulating,  relevant,  active,  and 
meaningful. 

Piaget  wrote: 

The  principal  goal  of  education  is  to  create  people  who  are  capable  of  doing  new 
things,  not  simply  repeating  what  other  generations  have  done  - people  who  are 
creators,  inventors,  discoverers.  The  second  goal  of  education  is  to  form  minds 
that  are  critical,  can  verify,  and  do  not  accept  everything  they  are  offered.  The 
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great  danger  today  is  from  slogans,  collective  opinions,  and  ready-made  trends  of 
thought.  We  have  to  be  able  to  resist  individually,  to  criticize,  and  to  distinguish 
between  what  is  proven  and  what  is  not.  So  we  need  [individuals]  who  are 
active,  who  learn  early  to  find  out  for  themselves,  partly  by  their  own  spontaneous 
activity  and  partly  through  the  materials  we  set  up  for  them;  who  learn  early  to 
tell  what  is  verifiable  and  what  is  simply  the  first  idea  to  come  to  them,  (as  cited 
in  Thomas,  Enloe,  & Newell,  2005,  pp.  41-42) 

Boyer  (1995)  contends  that 

. . .community  doesn't  just  happen. . .To  become  a true  community,  the  institution 
must  be  organized  around  people. . .What  we  are  really  talking  about  is  the  culture 
of  the  school  [college] . . .the  vision  that  is  shared,  the  ways  people  relate  to  one 
another... Simply  stated,  the  school  [college]  becomes  a community  for  learning 
when  it  is  a purposeful  place,  a communicative  place,  a just  place,  a disciplined 
place,  a caring  place,  and  a celebrative  place,  (pp.  17-18) 

Directly  from  a higher  education  (HE)  perspective,  Eenning  and  Ebbers  (1999)  indicated 
that  learning  communities  are  one  of  the  hottest  topics  in  higher  education,  and  that  has 
continued  to  be  true.  They  provide  three  distinct  perspectives  on  learning  communities 
specific  to  higher  education:  most  common  - the  curricular  approach  linking  clusters  of 
classes  using  an  interdisciplinary  theme,  enrolling  a common  cohort  of  students.  Another 
perspective  exists  from  the  technology  arena,  where  technology  provides  the  vehicle  to 
link  students  and  faculty  through  the  Internet.  Einally,  internationally,  learning 
communities  link  people  from  different  countries.  Eenning  and  Ebbers  mention  Dewey 
and  his  principles  on  the  social  process  of  learning  (discussed  more  below).  Eearning 
communities  today  have  their  roots  in  cooperative  and  collaborative  learning,  where 
social  interaction  provides  the  opportunity  for  active  learning.  Eenning  and  Ebbers  feel 
learning  communities  use  the  best  principles  of  student  development;  therefore,  they  are 
focused  on  the  development  of  student  or  faculty/student  learning  communities. 

The  need  for  higher  education  learning  communities  to  improve  undergraduate  education 
and  to  move  teaching  methodology  beyond  the  lecture  approach  to  large  groups  of 
students  comes  out  of  the  call  described  above.  In  higher  education,  if  there  is  any  focus 
at  all,  it  is  on  teaching,  not  student  learning.  The  public  cry  is  to  maximize  student 
learning  and  hold  ourselves  accountable  for  improved  student  learning.  The  following 
recommendations  are  made  by  the  Boyer  Commission  for  Research  Universities  (1998) 
but  apply  equally  well  for  other  universities: 

1 . Make  research-based  learning  the  standard; 

2.  Construct  an  inquiry-based  first  year; 

3.  Build  on  the  freshman  foundation; 

4.  Remove  barriers  to  interdisciplinary  education; 

5.  Eink  communication  skills  and  coursework; 

6.  Use  information  technology  creatively; 

7.  Culminate  with  a capstone  experience; 

8.  Educate  graduate  students  as  apprentice  teachers; 

9.  Change  faculty  reward  systems;  and 
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10.  Cultivate  a sense  of  eommunity.  (p.  37) 

In  establishing  these  standards  and  the  six  prineiples  below,  the  attempt  is  to  reveal  a 
more  integrated  vision  of  eommunity,  foeusing  on  the  quality  the  students  eneounter 
while  enrolled  in  higher  edueation: 

1.  By  a purposeful  eommunity,  we  mean  a plaee  where  faculty  and  students 
share  academic  goals  and  work  together  to  strengthen  teaching  and  learning 
on  campus. 

2.  By  an  open  community,  we  mean  a place  where  freedom  of  expression  is 
uncompromisingly  defended  and  where  civility  is  powerfully  affirmed. 

3.  By  a just  community,  we  mean  a place  where  the  sacredness  of  each  person  is 
honored  and  where  diversity  is  aggressively  pursued. 

4.  By  a disciplined  community,  we  mean  a place  where  individuals  accept  their 
obligations  to  the  group  and  well-defined  governance  procedures  guide 
behavior  for  the  common  good. 

5.  By  a caring  community,  we  mean  a place  where  the  well-being  of  each 
member  is  sensitively  supported  and  where  service  to  others  is  encouraged. 

6.  By  a celebrative  community,  we  mean  a place  where  the  heritage  of  the 
institution  is  remembered  and  where  rituals  affirming  both  tradition  and 
changes  are  widely  shared.  (Boyer  Commission,  1998,  p.  6) 

This  sense  of  community  is  "an  intentional  means  to  a particular  end,  which  end  is  to 
maximize  learning  of  groups  and  individuals  within  those  groups"  (Boyer  Commission, 
1998,  p.  8).  A learning  community  is  intentionally  developed  to  promote  and  maximize 
learning.  Most  learning  communities  share  a common  feature  that  of  "shared 
knowledge. . .The  second  is  shared  knowing"  (Tinto,  1998,  p.  171).  It  is  suggested  that 
they 

• Incorporate  and  value  diversity, 

• Share  a culture, 

• Foster  internal  communication, 

• Promote  caring,  trust,  and  teamwork, 

• Involve  maintenance  processes  and  governance  structures  that  encourage 
participation  and  sharing  of  leadership  tasks, 

• Foster  the  development  of  young  people,  and 

• Have  links  with  the  outside  world.  (Gardner,  1989) 

Two  dimensions  of  learning  communities  might  be  important  to  higher  education. 
Primary  membership  differentiates  which  characteristics  group  members  have  in 
common.  Primary  form  of  interaction  establishes  whether  group  members  have 
connection  capability,  direct,  in  person  (physical)  interaction,  virtual,  or  other  types  of 
correspondence  interaction. 

Now  that  we  have  established  that  many,  if  not  most,  of  the  universities  addressing  the 
development  of  learning  communities  are  focused  on  student  or  faculty-student  LCs, 
Penning  and  Ebbers  (1999)  provide  a complete  guide  to  that  purposeful  end.  However, 
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they  do  qualify  and  define  categories  of  learning  communities:  learning  organizations, 
faculty  learning  communities,  and  student  learning  communities. 

• Learning  organizations  are  colleges  and  universities  "consciously  structured  to 
promote  their  own  learning  and  that  of  their  students  and  faculty  members,"  as 
contrasted  with  what  has  generally  been  the  case  - "shared  learning 
experiences"  that  seem  "to  promote  individual,  isolated,  passive  learning  and 
forms  of  discourse  that  are  [currently]  very  much  limited  to  the  narrow 
boundaries  of  separate  disciplines" 

• Faculty  learning  communities  are  consciously  and  proactively  structured 
faculty  groups  organized  primarily  to  promote  faculty  learning. 

• Student  learning  communities  are  consciously  and  proactively  structured 
student  groups  organized  to  promote  student  learning.  (Tinto,  1997b,  p.  2 as 
cited  in  Lenning  & Ebbers) 

Lewis  and  Allan  (2005)  identify  types  of  learning  communities  as  those  established  to 
disseminate  good  practice;  to  provide  the  opportunity  for  innovation  and  improved 
practices  as  well  as  for  international  collaboration  (in  our  case,  internal,  then  national, 
and  perhaps  internationally),  for  multi-professional  collaboration,  and  finally,  for  cross- 
sector collaboration  to  improve  performance  (p.  18).  I believe  the  above  goal  is  also 
appropriate  for  our  initiative.  In  addition,  Enloe,  in  Thomas,  Enloe,  and  Newell  (2005), 
itemize  a list  of  principles  that  they  honor  in  a learning  community:  1)  excellence  and 
hard  work;  2)  [positive]  attitudes  toward  learning;  3)  respect  for  diversity;  4)  social 
responsibility;  5)  egalitarianism;  6)  empathy;  7)  development  of  an  ethical  self;  8) 
commitment  to  an  environmental  view;  and  9)  an  appreciation  of  the  importance  of 
present  (p.  46).  Also  interesting  to  consider  is  their  operational  framework  credited  to 
Quaker  beliefs:  "our  1)  social  nature  results  in  2)  interactive  relationships,  guided  by  3) 
principles  of  democracy,  reflecting  4)  restorative  practices  such  as  5)  the  circle  process" 
(p.  59).  In  their  discussion  about  social  by  nature,  they  reference  Kropotkin  (1910),  who 
researched  insects,  birds,  mammals,  and  humans  a century  ago.  The  greater  the 
sociability  of  a species,  the  greater  its  intelligence;  the  greater  its  intelligence,  the  greater 
its  chance  for  survival.  Human  socializing,  relating,  and  communicating  led  to  becoming 
smarter.  Kropotkin  considers  relating  and  communicating  as  "being  social."  Therefore,  if 
we  are  social  in  nature,  why  have  we  isolated  ourselves  and  students  from  group  learning. 
As  we  all  know,  the  sum  of  the  group  knowledge  and  learning  is  exponentially  greater 
than  the  sum  of  individual  knowledge  and  learning.  Healthy  relationships  are  basic  to 
content. 

Learning  occurs  at  a higher  level  where  good  relationships  exist.  Dewey  (1916) 
established  that  learning  is  a social  process  and  that  social  processes  require  interaction. 
Wheatley  (1999),  on  organizational  leadership  as  a new  science,  wrote  that  "In  the 
quantum  world,  relationships  are  not  just  interesting;  to  many  physicists,  they  are  all 
there  is  to  reality"  (p.  36).  Capra  (1996)  establishes  interdependence  and  connectedness 
as 

the  basic  principles  of  ecology  and  use[s]  them  as  guidelines  to  build  sustainable 
human  communities . . .The  first  principle  he  proposes  is  interdependence. . . .All 
members  of  an  ecological  community  are  interconnected  in  a vast  and  intricate 
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network  of  relationships,  the  web  of  life.  They  derive  their  essential  properties 
and,  in  fact,  their  very  existence  from  their  relationships  to  other  things. 
Interdependence  - the  mutual. . .dependence  of  all  life  processes  on  one  another— 
is  the  nature  of  all  ecological  relationships,  (pp.  61-62) 

Furthermore,  all  human  behavior  and  failure  or  success  of  it  is  interdependent  with  that  of 
the  others  in  the  community.  Therefore,  relationships  are  basic  to  the  content  and 
interaction  of  community.  Capra  (1996),  Dewey  (1916),  Wheatley  (1999)  clearly  state 
that  establishing  a community  value  system  and  democratic  processes  have  great  results. 
Their  values  for  students  were  reform,  project-based  learning,  accountability,  and 
democracy  as  a way  of  life:  democracy  in  the  value  of  the  individual  and  the 
simultaneous  value  of  community.  If  a conflict  comes  into  play  between  the  two, 
remember  to  recognize  the  value  of  both  the  individual  through  project-based  learning 
and  the  value  of  the  group  where  individuals  are  members  (Dewey).  Although  their  focus 
for  developing  the  four  skills:  1)  speaking,  2)  listening,  3)  making  positive  decisions  for 
self,  and  4)  making  positive  decisions  with  others  was  for  students,  these  skills  are 
required  of  those  at  any  age  or  generation  if  they  are  to  be  successful.  Principles  of 
restorative  practice  stem  from  restorative  measures  in  the  criminal  justice  system,  where 
a change 

represents  a philosophic  shift.  For  some,  this  may  be  a shift  from  perceiving 
some  students  as  a problem  to  actually  being  able  to  identify  their  strengths  and 
potential  for  success.  For  others,  it  may  be  a change  in  how  behavioral  problems 
are  handled.  Expanding  or  beginning  restorative  measures  means  getting  everyone 
to  agree  with  the  idea  of  restoration  rather  than  punishment  or  control  models  of 
behavioral  management.  Restorative  measures  also  mean  respecting  that  all 
affected  parties  have  the  ability  to  contribution  to  the  solution.  Once  people  are 
informed  and  understand  the  guiding  principles,  they  can  begin  to  think  of  a 
myriad  of  applications.  (Minnesota  Department  of  Children,  Families  and 
Learning,  1996,  n.p.) 

Although  this  was  applied  to  children,  it  has  meaning  for  us  in  this  initiative.  We  have 
perceptions  about  students,  and  how  we  interact  with  them  is  critical  to  their  learning, 
regardless  of  age.  Restorative  practices  include  cooperation,  showing  respect,  being 
authentic,  interacting  with  the  student,  speaking  with  and  listening  to  the  student, 
supporting  students’  self  control  and  efforts  to  solve  problems,  encouraging  the  use  of 
head  and  heart,  thinking  and  feeling,  focusing  on  learning,  and  using  group  processes. 
This  can  be  valuable  later  in  our  student  learning  section,  but  also  speaks  to  how  we 
should  treat  each  other.  To  them,  the  circle  process  entails  being  open.  For  us,  the  circle 
also  represents  a closed  loop  process  where  feedback  provides  information  for  changes, 
and  when  that  information  results  in  a change,  the  loop  has  been  appropriately  and 
productively  closed  by  taking  action. 

Thomas,  Enloe  and  Newell  (2005)  go  on  to  consider  the  work  by  Collay,  Dunlap,  Enloe, 
and  Gagnon  (1998)  on  learning  circles,  where  the  definition  of  a learning  circle  is 

small  communities  of  learners  among  teachers  [professors]  and  others  [those 
external  to  schools  or  universities  active  in  communities  of  practice]  who  come 
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together  intentionally  for  the  purpose  of  supporting  each  other  in  the  process 
of  learning. . . .[agreeing]  that  the  purpose  of  life  is  learning  - and  that  learning 
is  constructivist  for  all  living  systems,  (p.  ix) 

Collay  et  al.  (1998)  identify  six  critical  conditions  for  learning  communities:  1)  building 
community;  2)  constructing  knowledge;  3)  supporting  learners;  4)  documenting 
reflection;  5)  assessing  expectations;  and  6)  changing  cultures  (p.  x).  These  conditions  set 
the  process  and  content  to  be  constructivist.  They  define  "communities  of  learners  as 
. . .groups  of  people  gathered  together  intentionally  for  the  purpose  of  supporting  each 
other  in  the  process  of  learning. . .e.g.,  college  faculty,  universities,  schools,  or  districts," 
business  or  industry  (p.  xv).  Smaller  groups  of  learners  within  the  larger  group  or 
community  are  learning  circles  in  which  there  is  a "more  personal  forum  for  professional 
interaction  and  greater  opportunity  for  conversations,. . .where  members  can  construct 
ideas  together... share  opinions.... and  debate  issues"  (p.  xvi).  Learning  circles  and 
communities  provide  a framework  where  individuals  can  affect  change  that  results  in 
influencing  their  organizations.  Members  have  choices,  take  responsibility,  and  then 
work  with  leaders  to  set  an  agenda.  These  authors  identify  four  factors  for  professional 
cultures  directly  related  to  student  achievement:  1)  shared  decision  making;  2)  a shared 
sense  of  purpose;  3)  collaborative  work  toward  that  purpose;  and  4)  collective 
responsibility.  Learning  circles  can  provide  a framework  for  educators  to  change  their 
practice,  thus  influencing  the  organization  within  which  they  perform.  They  believe  that 
"reform  is  done  to  an  organization  [our  college  and  departments],  but  organizations  don't 
think  or  change;  individuals  do"  (p.  xvi).  Learning  circles  and  communities  offer  the 
opportunity  for  a critical  mass  to  come  together  to  make  meaning  of  an  organizational 
structure  or  culture,  through  membership,  and  then  take  action,  engage  in  change,  to 
influence  the  organization.  Learning  circles  make  up  a broader  learning  community,  but 
they  are  a distinct  group  of  people  who  engage  in  learning  and  action  together,  entering 
into  dialogue,  making  decisions,  trying  new  ideas  or  strategies.  Learning  circles,  and 
possibly  the  community  if  it  too  is  within  a broader  organizational  context,  do 
differentiate  themselves  from  the  broader  community  and  usually  from  other  types  of 
activities  that  might  be  external,  yet  related,  to  their  endeavors.  Their  power  to  support 
each  other  is  an  intrinsic  aspect  of  the  social  and  organizational  "fabric."  Generally,  these 
circles  or  communities  are  informal,  although  formal  structures  and  organizations  can 
create  a context  for  their  work.  Their  work  is  based  upon  similar  interests,  and  they  have 
usually  interacted  with  each  other  within  their  professional  context. 
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Figure  A. 6. 2:  Community  of  Learners  Schematic 


Communities  of  Learners  Schematic 


rij^urf  1.1,  C'uminuniUc.s  ol  Learners  Scheifii tic 

(Collay  et  ah,  1998,  p.  4) 

Collay  et  al.  (1998)  provide  a good  synthesis  of  theoretical  background  on  learning 
communities  and  circles.  They  believe,  in  sync  with  Senge  (1990,  1999),  that  learning  is 
a process  of  change  and  that  when  learning,  change  occurs.  It  happens  when  an  individual 
or  group  constructs  "patterns  of  action  to  solve  problems  of  meaning"  (Collay  et  ah,  p.  3). 
Humans  experience  four  environmental  dimensions:  matter,  energy,  space,  and  time  and 
then  construct  patterns  of  action  in  each  dimension.  Those  patterns  could  be  across  the 
physical,  symbolic,  social,  and  theoretical  and  take  place  by  "coordinating  movements, 
establishing  routines,  inventing  realities,  answering  questions,  making  decisions,  setting 
expectations,  creating  metaphors,  and  building  theories"  (p.  3).  Usually  perspectives 
differ  across  individuals.  Collay  et  al.  identified  six  theoretical  strands  from  which  they 
based  their  collective  thought  and  theoretical  foundation:  1)  living  organisms;  2) 
constructivist  learning;  3)  group  process;  4)  complex  systems;  5)  optimal  experience;  and 
6)  interdependent  networks.  The  figure  below  shows  the  relationships  between  the 
theoretical  threads  and  the  six  essential  conditions,  also  identifying  the  theorists.  We  used 
as  a primary  guide  to  our  forming  and  action  process. 
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Table  A.6.1:  Theoretical  Overview 
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(Collay  et  ah,  1998,  p.  6) 


Although  many  of  the  sources  on  learning  communities  focus  on  their  development  at  the 
K-12  level,  this  does  not  mean  that  their  strategies,  designs,  processes,  techniques  and 
methods  do  not  apply  to  the  post  secondary  level.  Seemingly,  K-12  has  engaged  in 
change  that  also  applies  to  higher  education,  and  in  my  humble  opinion,  this  often 
happens  when  higher  education  should  have  been  leading  the  design  or  development  of 
educational  change  in  partnership  with  K-12  or  should  have  been  doing  the  initial 
research  and  then  leading  K-12  to  do  so. 

Wilson  and  Ryder  (1996)  define  dynamic  learning  communities  as  communities  in  which 
“members  share  control  and  everyone  learns,  including  the  facilitator  or  tutor  or  group 
leader.  Transformative  communication  is  the  norm,  with  both  sender  and  receiver  of 
messages  changed  by  the  interaction. . .all  participants  are  engaged  in  the  learning 
experience”  (p.  11).  Lewis  and  Allan  (2005)  define  a learning  community  as  "a 
supportive  group  of  people  who  come  together  to  collaborate  and  learn  together;  they  are 
usually  facilitated  or  guided  to  achieve  a specific  outcome  or  agreed  learning  objectives" 
(p.  8).  They  discuss  "transformative"  learning  communities  as  those  that  enable  "like- 
minded  people  or  colleagues  or  professionals  with  a common  or  multi-professional 
interest  to  work  together  and  to  achieve  a particular  aim  or  organizational  objective"  (p. 
6).  They  go  on  to  characterize  learning  communities  as  where  there  is 

• a shared  goal,  problem  or  project; 

• shared  resources; 

• shared  membership  and  leadership; 

• commitment  to  improvement  of  professional  practice; 

• collaborative  approaches  to  group  work; 

• learning  and  development  focused  on  real  work-based  issues  and  practice; 
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• autonomous  community  members; 

• high  levels  of  dialogue,  interaction  and  collaboration; 

• information  and  knowledge  sharing; 

• knowledge  constructivism; 

• knowledge  transfer  and  knowledge  exchange; 

• use  of  information  and  communication  technologies,  (pp.  6-7) 

Learning  communities  may  exist  within  organizations  to  engage  and  empower  a creative 
group  of  colleagues  with  "complementary  set  of  knowledge,  skills  and  attitudes  to 
produce  dynamic  approaches  to  problem  solving,  knowledge  management  and 
knowledge  creation"  (Lewis  & Allan,  2005,  p.  6).  Often,  as  with  our  initiative,  there  is  a 
structured  program  of  study.  With  others,  as  also  with  our  initiative,  communities  are 
established  to  manage  change  and  deal  with  any  related  issues  pro-actively.  A program  of 
learning  and/or  training  may  be  characterized  as: 

• pre-planned  with  pre-determined  aims  and  learning  outcomes; 

• program  is  often  owned  by  an  education  or  training  group  (not  true  for  ours; 
we  have  developed  our  own) 

• programs  are  facilitated  by  a tutor  or  trainer  who  has  a responsibility  for 
curriculum  content  and  ensuring  that  the  program  follows  its  pre-determined 
course,  including  a set  range  of  activities  or  tasks; 

• clear  start  and  end  date; 

• differential  in  power  between  the  learner  and  the  tutor/trainer; 

• programs  are  often  accredited,  for  example  by  higher  education  institutions  or 
professional  bodies; 

• the  learner  is  sometimes  viewed  as  the  'customer'  who  consumes  the  learning, 
(p.  5) 

Some  of  these  characteristics  fit  our  initiative  and  some  do  not.  We  do  have  pre- 
determined aims  and  outcomes;  there  is  a facilitator  for  which  some  power  distance  is 
true;  there  is  a clear  start  and  end  date;  and  the  learners  are  our  own  faculty  members. 
Some  of  the  characteristics  above  better  describe  the  relationship  that  might  occur 
between  K-12  schools  and  higher  education  (Lewis  & Allan,  2005).  Therefore,  our 
learning  community  is  meeting  the  criteria  identified  here  as  well  as  other  criteria  to  be 
mentioned  below  and  will  engage  in  a program  of  learning  and  experimentation  on 
teaching  and  learning. 

Important  to  any  learning  community  discussion  is  the  community  of  practice  (CP) 
within  which  it  exists.  Characteristics  of  a community  of  practice  (professional  real  world 
groups  or  organizations  within  which  groups  engage)  are 

• common  purpose  identified  by  participants; 

• shared  membership  and  leadership; 

• participants  likely  to  be  at  different  stages  in  their  professional  life; 

• development  of  professional  practice  through  apprenticeship; 

• acceptance  of  low  levels  of  participation  by  new  members,  that  is,  legitimate 
peripheral  participation; 
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• development,  creation  and  management  of  knowledge  within  organizations; 

• open-ended,  not  time  bound; 

• importance  of  dialogue,  interaction  and  shared  narratives.  (Lewis  & Allan,  2005, 
p.  7) 

To  follow  our  logic,  it  is  important  to  understand  another  community,  that  of  community 
of  interest.  These  involve  larger  groups  of  people,  perhaps  a network  or  larger  body. 

They  may  support  the  dissemination  and  exchange  of  information  but  probably  are  not 
directly  supportive  of  the  collaborative  learning  processes  that  occur  between  members  of 
the  LPs  or  LCs.  An  example  of  a Cl  is  those  members  of  the  National  Association  of 
Industrial  Technology;  the  American  Society  of  Engineering  Educators;  those  involved  in 
the  discipline  specific  organizations  such  as  Plastics  Engineering,  the  Society  of 
Manufacturing  Engineers,  etc.;  and  groups  of  members  who  cluster  together  around 
specific,  but  common  or  related,  interests.  In  my  opinion,  a community  of  interest  to  our 
initiative  would  be  the  broader  university  community  within  which  we  are  organized  as  a 
college  of  four  departments.  The  Table  below  presents  a comparison  succinctly: 

Eigure  A. 6. 3:  Eearning  Community  Relationships 


Figure.  1. 1 Relationships  between  learning  programmes,  learning  communities  and 
communities  of  practice 
(Eewis  & Allan,  2005,  p.  8) 
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Table  A. 6. 2:  Comparison  of  Learning  Communities 


Table  1.1  t .ompaiistin  ol  learning  coinmunilie>i  and  communities  of  iiueresi 
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(Lewis  & Allan,  2005,  p.  9) 


Lewis  and  Allan's  (2005)  figure  implies  an  overlap  or  integration  between  the  learning 
community,  community  of  practice,  and  learning  program.  Figure  1.1  and  Table  1.1 
indicates  that  they  seem  to  use  the  terms  “interest”  and  “practice”  interchangeably.  The 
CELT  model  acknowledges  the  integration  or  overlap,  but  in  a different  way.  Our  model 
reveals  more  of  a nesting  relationship  between  entities.  There  is  a clear  relationship 
between  the  four  Scholarships  identified  by  Boyer  (1990);  and,  the  Scholarship  of 
Teaching  is  a manifestation  of  each  of  the  other  models  as  well  as  a type  of  research  on 
its  own  merit.  The  research  context  is  one  of  learning  for  we  acknowledge  the  College  of 
Engineering  and  Engineering  Technology  as  a formal  Teaming  Organization  (Senge, 
1990).  The  faculty  EC  for  this  initiative  is  a smaller  learning  organization,  but  a formal 
one,  within  the  college  framework,  we  hope  other  ECs  evolve  within  the  college  context, 
the  EC  is  interdisciplinary  and  exists  outside  the  formal  departments;  however,  the  model 
acknowledges  the  relationships  between  the  four  departments.  It  was  our  hope  to  engage 
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four  “learning  circles”,  one  from  each  department,  and  that  did  occur  for  three 
departments.  However,  one  department  lost  a member,  therefore,  there  was  only  one 
individual  from  that  particular  department.  We  prefer  the  model  of  departmental  learning 
circles  that  then  make  up  the  interdicsiplinary  college  Learning  Communities  of  faculty 
members.  The  college  exists  within  the  university  and  the  community  of  practice  of  all 
universities.  It  also  exists  within  the  “real-world”  context  of  the  disciplinary 
communities  of  practice,  business  and  industry.  Yet,  broader  than  those,  are  the 
communities  of  interests,  e.g.  State  of  Illinois  and  its  Citizens  Agenda  for  state 
universities.  And  finally,  we  must  acknowledge  the  broader  global  communities  of  both 
practice  and  interest.  Of  course,  we  agree  with  Lewis  and  Allan  that  everything  overlaps 
and  is  integrated,  but  we  also  want  to  reflect  the  nesting  of  one  environment  within 
another.  That  is  also  why  our  faculty  development  program  was  designed  as  an 
integrated  program,  rather  than  a separated  set  or  series  of  workshops;  also,  that  is  why  it 
began  with  building  a foundation  and  extended  to  include  implementation  and  research 
AS  an  INTEGRAL  ASPECT  of  the  program  design.  The  CEET  Model: 
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Figure  A. 6. 4:  CEET  Eearning  Community  Model-Northern  Illinois  University 


Global  Communities  of  Practice  and  Interest 


Regional  & National  Public  Communities  of  Interest 


Business  & Industry  Communities  of  Practice 


(Scarborough,  2006) 
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Furthermore,  Lewis  and  Allan  (2005)  educate  us  about  virtual  learning  communities 
(VLC).  These  can  be  powerful  because  they  can  release  us  from  the  geographic,  cultural, 
and  demographic  constraints  of  the  thoughts,  practices,  considerations,  and  actions.  They 
identify  why  there  is  a rise  in  virtual  learning  communities  beyond  technological 
advancements: 

• intensifying  competition  and  globalization  ; 

• new  ways  of  working,  for  example  collaborative  partnerships; 

• accelerating  change  and  the  need  to  manage  change  and  problem  solve; 

• information  explosion  and  the  rise  of  knowledge  management; 

• developing  and  converging  communication  and  information  technologies; 

• need  for  continuous  professional  development,  (p.  11) 

Virtual  LCs  allow  us  to  work  together,  support  each  other,  and  work  when  it  suits  our 
time,  so  we  can  maintain  some  level  of  work- life  balance.  Important  to  note  is 
The  information  explosion  continues  in  size,  complexity  and  diversity. 
Information  overload  is  a recognized  stress  stimulus  and  our  business  is  in 
managing,  organizing  and  enabling  the  exploitation  of  information,  so  that  we 
have  control  and  contain  the  explosion  on  behalf  of  organizations  and  individuals 
who  need  or  demand  not  overload  but  filtered,  validated  and  authoritative 
information.  (Lewis  & Allan,  2005,  p.  12) 

Although,  we  are  not  a virtual  community,  we  did  try  to  contain  the  information  overload 
of  our  participating  faculty  members  by  doing  tasks  one  time  for  all  - for  example,  this 
learning  document.  It  was  our  intent  to  eliminate  redundant  activity,  make  things  as 
simple  but  illuminating  as  possible  in  designing  and  developing  the  foundation  for 
learning  and  action.  We  established  existing  knowledge,  explored  the  views,  theories,  and 
thoughts  of  others  to  build  or  extend  "explicit"  knowledge,  and  then  acted  upon  that 
knowledge  through  research  with  students,  also  extending  "tacit"  knowledge.  Tacit 
knowledge  is  built  through  personal  experience,  whereas  explicit  knowledge  can  be 
codified  and  explained.  Learning  communities  engage  in  the  key  activity  of 

transforming  tacit  into  explicit  knowledge  [by] . . .explor[ing]  experience  and 
practice,  interrogat[ing]  new  ideas,  concepts  and  evidence,  discuss [ing]  personal 
perspectives  and  concerns  with  fellow  professionals. . .interactions  often  result  in 
the  development  of  clarity  and  a shared  language. . .LCs  provide  a vehicle 
whereby  new  members. . .are  able  to  tap  into  the  tacit  and  explicit  knowledge  of 
experienced  practitioners.  (Lewis  & Allan,  2005,  pp.  12-13) 

This  can  occur  whether  a VLC  or  not.  Managing  knowledge  can  take  place  through  three 
approaches:  1)  databases  and  repositories;  2)  knowledge  route  maps  and  directors;  and  3) 
knowledge  communities  and  networks.  Technology  as  it  is  defined  above  is  "the  science 
of  efficient  action  that  extends  human  potential  and  capacity";  thus  “individual  and 
collective  power,  ultimately  transforming  society  and  our  world”  provides  the 
technological  capability  of  creating  LCs  across  the  world  because  now  we  can 
communicate,  if  careful,  quite  effectively  ((Lewis  & Allan,  2005,  p.  14).  Although  Lewis 
and  Allan  focus  on  virtual  learning  communities,  the  lists  of  benefits  of  belonging  to  a 
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VLC  and  benefits  to  one's  continuing  professional  development  (CPD)  also  apply  to  our 
initiative 

Figure  A. 6. 5:  Benefits  Checklist  - Virtual  Learning  Communities 
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(Lewis  & Allan,  2005,  p.  17) 

Figure  A. 6. 6:  Benefits  Checklist  - Organizations 


Box  1.3  Checklist  of  benefits  for  organizations 
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(Lewis  & Allan,  2005,  p.  18) 
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Although  Smith,  MacGregor,  Matthews  and  Gabelnick  (2004)  focus  on  student  learning 
communities,  they  offer  core  practices  for  learning  communities  for  which  the  challenge 
is  to  "take  advantage  of  the  learning  community  structure  to  capture  and  intensify  the 
synergistic  possibilities  for  meaningful  community  building  and  learning. . .spaces  to 
bring  together  the  theory  and  practice  of  student  [faculty]  development  and  diversity,  of 
active  inclusive  pedagogies,  and  of  reflective  assessment"  (p.  97).  They  recommend  that 
all  LCs  be  implemented  with  these  core  practices  in  mind:  community,  diversity, 
integration,  active  learning,  and  reflection/assessment. 

Figure  A. 6. 7:  Core  Practices 


Figure  4.1.  Core  Practices  in  Learning  Communities 


(Smith  et  al.,  2004,  p.  98) 

Their  model  is  based  upon  inclusion,  where  learners  feel  a sense  of  belonging  and 
connectedness,  both  academically  and  socially.  Faculty  teaching  teams  are  one  way  to 
enhance  this  for  both  students  and  faculty  members,  establishing  relationships  between 
courses,  in-class  and  external  activities,  general  education  and  major  components  of  the 
program,  informal  social  occasions,  study  groups,  and  service  learning  projects.  An 
example  of  service  learning  in  one  of  our  classes  is  where  student  teams  are  expected  to 
seek  out  a service  project  in  the  community  that  requires  leadership  and  use  of  their 
collective  knowledge  and  skills,  provides  a service,  and  inherently  helps  prepare  them  to 
become  a formal  team.  This  works  for  the  course  content,  engages  students  in  getting  to 
know  each  other,  helps  them  to  realize  that  corporations  expect  them  to  provide 
community  leadership,  and  so  much  more.  Service  learning,  although  not  a new  concept, 
is  a renewed  practice  and  can  be  a very  effective  teaching  method,  serving  for  knowledge 
gain  as  well.  Collaboration  and  interdependence  occur  when  learners  learn  in  community 
or  engage  in  collaborative  work  to  make  meaning,  debate  issues,  solve  problems,  create 
products,  or  engage  in  other  learning  tasks.  They  have  to  practice  and  become  more 
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skilled  at  articulation,  listen  to  others,  consider  other  ideas  than  their  own,  contribute  to 
and  negotiate  (work  through)  constructive  disagreements  or  conflicts  - learn  to  work 
together,  a vital  skill  today!  There  will  be  challenges.  However,  this  is  required  for  one 
active  in  a community  of  practice  as  discussed  above,  where  knowledge  sharing  occurs 
for  a result  of  growth.  Learners  are  diverse  in  background  (e.g.,  educational  preparation 
and  experience,  practical  or  work  experience,  personalities,  work  styles,  information 
processing  styles,  culture,  socio-economically,  politically,  and  in  so  many  more  ways) 
whether  faculty  or  students.  Diversity  lends  itself  to  more  significant  results  as  long  as 
participation  by  all  is  the  practice. 

Most  important  is  ways  of  knowing  (and  learning).  If  the  instructional  leader  is  analytical 
and  focuses  more  in  intellectual  concepts  while  their  learners  favor  experiential  or 
relational  forms  of  learning,  there  may  be  a conflict.  The  Association  of  American 
Colleges  (1982)  found  that  direct  lecture-centered  courses  were  not  as  successful  with 
women.  Belenky,  Clinchy,  Goldberger,  and  Tarule  (1986)  provide  an  influential  book  on 
women  and  their  ways  of  knowing:  women  prefer  more  connected  ways  of  learning  with 
more  dialogue  where  life  experiences  are  part  of  the  process  and  where  they  can  find 
their  voices  and  feel  validated  through  connections  with  new  knowledge  and 
understanding.  More  has  been  done  since  this  work  on  Latinos  and  African-American 
students  (Tierney,  Coylar,  & Corwin,  2003). 

Integration  is  a model  component  meaning  "continually  blend[ing]  the  old  and  the 
new. . .We  construct  our  understanding  using  part  of  what  we  already  know  and  part  of 
what  is  new"  (Zull,  2002,  p.  1 19).  Deep  learning  occurs  best  when  creating  structures  and 
opportunities  to  stimulate  connection  making,  extending  one's  schema  or  framework  of 
knowledge,  and  providing  opportunities  to  change  it  with  new  knowledge  or  a 
combination  of  the  old  and  new.  We  can  add  new  or  change  the  old  by  combining  it  with 
the  new. 

• Surface  learning  refers  to  short  term  memory  and  the  information  embedded 
in  the  short  term  memory;  surface  learning  is  usually  not  internalized  to 
become  part  of  one's  knowledge  structure,  or  schema. 

• Deep  learning  is  that  which  is  more  permanently  embedded  in  one's  evolving 
understanding  of  a subject;  it  transforms  the  learner  in  some  way.  These  can 
provide  an  orientation  to  learning  (e.g.,  memorizing  and  reproducing  from 
recall  versus  where  one  makes  more  personal  meaning  by  transforming 
information  and  ideas  in  relation  to  what  they  already  know  - their  knowledge 
and  experience)  (Entwistle,  2000,  p.  10;  Marton  as  cited  in  Ramsden,  1992). 

• Active  learning,  another  component  of  the  model,  or  the  constructive  process, 
occurs  when  learners  engage  in  an  experiential  process  that  allows  for  the 
attaching  of  new  knowledge  to  what  one  already  knows,  making  and  remaking 
one's  meaning  or  understanding.  With  new  knowledge,  an  old  bit  or 
understanding  changes  into  something  new.  Knowledge  is  forever  changed  if 
it  goes  beyond  the  "incorporation  of  inert  ideas  merely  received  into  the  mind 
without  being  utilized,  or  tested,  or  thrown  into  fresh  combinations" 
(Whitehead,  1929/1949,  p.  13)  and  promoting  active  learning  where 
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learning  is  not  a spectator  sport.  Students  [faculty]  do  not  learn  much  just 
by  sitting  in  class  listening  to  teachers,  memorizing  prepackaged 
assignments,  and  spitting  out  answers.  They  must  talk  about  what  they  are 
learning,  write  about  it,  relate  it  to  past  experiences,  apply  it  to  their  daily 
lives.  They  must  make  what  they  learn  part  of  themselves.  (Chickering  & 
Gamson,  1987,  p.  3) 

• Finally,  the  last  component  for  an  LC  is  reflection  and  assessment,  where 
assessment  occurs  as  learning.  This  will  be  discussed  in  more  depth  later,  but 
presently  let  us  say  that  it  requires  learners  to  reflect  upon  their  work  and 
consider  what  matters,  where  they  began,  where  they  are,  and  what  is  still  to 
be  accomplished.  In  further  considering  reflection  and  assessment,  it  can 
occur  in  three  ways:  assessment  as  learning,  reflection  as  learning,  and 
communities  of  reflection. 

- Assessment  as  learning  improves  work  over  time,  "a  process  integral  to 
learning  that  involves  observation  and  judgment  of  each  student's 
[professor's]  performance  on  the  basis  of  explicit  criteria  with  resulting 
feedback"  (Alverno  College  Faculty,  1994,  p.  3).  The  focus  is  learning,  and 
the  developmental  improvement  of  that  learning  is  the  center  of  the 
assessment  practice.  Work  is  compared  against  criteria,  expectations,  or 
standards. 

- Reflection  as  learning  is  where  students  are  asked  to  examine  their  own 
prior  knowledge  and  assumptions  explicitly  and  pay  attention  to  how  the 
new  learning  is  changing  those  assumptions,  understandings,  or  meanings. 
Students  identify  where  they  are  making  new  connections/meanings,  or 
where  their  understandings  have  changed. 

- Communities  reflection  is  where  collaborative  learning  has  taken  place 
and  where  the  group  reflects  together,  processing  as  a collective,  extending 
their  understandings  and  meanings  to  include  those  of  others;  then  may, 
again,  change  their  own.  When  looking  back,  integration  and  closure  to  the 
academic  and  social  learning  experience  has  been  achieved. 
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Figure  A. 6. 8:  The  Logical  Structure  of  Approaches  to  Learning 


APPROACH  TO  LEARNING 


'Structural'  aspect: 
the  act  of  experierx:ing, 
of  organising,  of  structuring 


that  which  is  experienced; 
the  significance  of  the  task 


HOUSTIC 
Preserves  the 
structure,  focuses 
on  the  whole  In 
relation  to  the  parts 


ATOMISTIC 
Distorts  the  structure, 
focuses  on  the  parts, 
segments  the  whole 


DEEP 
Focuses  on 
what  the  task  is 
atXKit  (e.g.  the 
author's  intention) 


SURFACE 
Focuses  on  the 
‘signs'  (e.g.  the 
word-sentence 
level  of  the  text) 


(Marton  as  cited  in  Ramsden,  1992) 


Table  A. 6. 3:  Different  Approaches  to  Learning 

Deep  approach 

Intention  to  understaad..  Student  maintains  structure  of  task. 

Focus  on  ‘what  is  signified’  (e.g.  the  author's  argument, ' 
or  the  concepts  appiicable  to  soMng  the  problern)'^^ 

Relate  previous  knowiedge  to  new  knowiedge.. 

Relate  knowledge  from  different  courses. 

Relate  theoretical  ideas  to  everyday  experience.. 

Relate  and  distinguish  evidence  and  argument.  . 

Organise  and  structure  content  into  a coherent  whole. 
internal  emphasis:  'A  window  through  which  aspects  of  reality  ' tg; 

become  visible,  and  more  intelligible’  (Entwistie  and  Marton.  .1984). 


Surface  approach 

Intention  only  to  complete  task  requirements. 
Student  distorts  structure  of  task. 


Focus  on  ’the  signs'  (e.g.  the  words  and  sentences  of  the  text,' or; 

unthinkingly  on  the  formula  needed  to  solve  the  problem).  - 

Focus  on  unrelated  parts  of  the  task; 

Memorise  information  for  assessments; 

Associate  facts  and  concepts  unrefiectively. 

Fail  to  distinguish  principles  from  examples. 

Treat  the  task  as  an  external  imposition.  . - ’ 

Exfema/ emphasis:  demands  of  assessments,  knowledge  cut  off  from.^ 


(Ramsden,  1992,  p.  46) 
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Table  A. 6.4:  Examples  of  Questions  in  the  Lancaster  Approaches  to  Studying  and  the 
Biggs  Study  Process  Questionnaires 

Orientation  to  studying  Indicative  items 

(Genera!  approach  to  learning) 


Meaning  orientation 
(Deep  approach) 


Reproducing  orientation 
(Surface  approach) 


(Ramsden,  1992,  p.  52) 


I try  to  relate  ideas  in  one  subject 
to  those  in  others,  whenever 
possible. 

I usually  set  out  to  understand 
thoroughly  the  meaning  of  what  I 
am  asked  to  read. 

In  trying  to  understand  new  ideas,  I 
often  try  to  relate  them  to  real-life 
situations  to  which  they  might  apply. 
When  I’m  tackling  a new  topic,  I 
often  ask  myself  questions  about  it 
which  the  new  information  should 
answer. 

In  reading  new  material  I often  find 
that  I’m  continually  reminded  of 
material  I already  know  and  see  the 
latter  in  a new  light. 

I spend  a lot  of  my  free  time  finding 
out  more  about  interesting  topics 
which  have  been  discussed  in 
classes. 

> 

I find  I have  to  concentrate  on 
memorising  a good  deal  of  what  we 
have  to  learn. 

1 usually  don’t  have  time  to  think 
cibout  the  implications  of  what  I 
have  read. 

Although  I generally  remember  facts 
and  details,  I find  it  difficult  to  fit 
them  together  into  an  overall  picture. 
I find  I tend  to  remember  things  best 
if  I concentrate  on  the  order  in 
which  the  lecturer  presented  them. 

I tend  to  choose  subjects  with  a lot 
of  factual  content  rather  than 
theoretical  kinds  of  subjects. 

I find  it  best  to  accept  the 
statements  and  ideas  of  my 
lecturers  and  question  them  only 
Uh^/  special  circynri^taiices. 


Dufour  and  Baker  (1998),  although  discussing  LCs  in  schools,  do  inform  us  about 
process  and  content.  They  begin  with  statements  from  Covey  (1996)  and  others.  Here  are 
a few  they  value:  "Only  the  organizations  that  have  a passion  for  learning  will  have  an 
enduring  influence"  (p.  149).  And  Drucker  (1992)  "Every  enterprise  has  to  become  a 
learning  organization.  Organizations  that  build  in  continuous  learning  in  jobs  will 
dominate  the  twenty-first  century"  (p.  109).  These  comments  certainly  apply  to  higher 
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education  and  schools  as  well.  Also  "If  schools  want  to  enhance  their  organizational 
capacity  to  boost  student  learning,  they  should  work  on  building  a professional 
community  that  is  characterized  by  shared  purpose,  collaborative  activity,  and  collective 
responsibility  among  staff’  (Newmann  & Wehlage,  1995,  p.  37).  And  finally,  "The  new 
problem  of  change. . .is  what  would  it  take  to  make  the  educational  system  a learning 
organization  - expert  at  dealing  with  change  as  a normal  part  of  its  work,  not  just  in 
relation  to  the  latest  policy,  but  as  a way  of  life"  (Fullan,  1993,  p.  14). 

In  beginning  with  the  end  in  mind  as  Covey  (1996)  suggests,  Dufour  and  Baker  (1998) 
define  what  an  LC  looks  like: 

1 . shared  mission,  vision,  and  values; 

2.  collective  inquiry  where  the  "team  learning  wheel"  is 

- public  reflection, 

- shared  meaning 

- joint  planning 

- coordinated  action; 

3.  collaborative  teams; 

4.  action  orientation  and  experimentation; 

5.  continuous  improvement  where  the  key  questions  are:  'What  is  our 
fundamental  purpose?  What  do  we  hope  to  achieve?  What  are  our  strategies 
for  becoming  better?  And  what  criteria  will  we  use  to  assess  our  improvement 
effort?;  and, 

6.  results  orientation  where  the  results  are  assessed  based  upon  results  rather 
than  intentions,  (pp.  26-29) 

Dufour  and  Baker  (1998)  model  the  corporation  model  and  value  the  strategies  of 
corporate  leaders.  They  go  further  to  suggest  four  building  blocks  of  a professional  BC: 

• Mission:  Why  do  we  exist?  Bverybody  can  learn  based  on  their  ability  if  they 
take  advantage  of  the  opportunity  to  learn,  and  we  will  accept  responsibility 
for  ensuring  their  growth  and  establish  high  standards  of  learning  that  we 
expect  all  students  to  achieve. 

• Vision:  What  do  we  hope  to  become? 

• Values:  How  must  we  behave  in  order  to  make  our  shared  vision  a reality? 

• Goals:  Which  steps  will  we  take  first,  and  when?  (pp.  60-62) 

Dufour  and  Baker  (1998)  suggest  that  sustainability  be  addressed  by  auditing  the 
effectiveness  of  the  BC  and  identifying  significant  factors  to  monitor  and  assess.  They 
consider  critical  questions  a good  process  and  the  timing  of  the  assessment  strategy  and 
make  a good  point  that  "Collaboration  by  Invitation  Does  Not  Work"  (p.  118).  The 
isolation  of  teachers,  for  them,  and  professors,  true  for  us,  is  so  ingrained  that  invitations 
for  collaboration  may  not  result  in  action;  therefore,  "to  build  professional  learning 
communities,  meaningful  collaboration  must  be  systematically  embedded  into  the  daily 
life  of  the  school"  (p.  118).  Although  they  are  speaking  of  K-12  situations  and  we  are  at 
the  university  level,  this  too  has  a lesson  for  us  because  professors  are  even  more  isolated 
and  independent  than  K-12  teachers.  Therefore,  collaboration  has  to  be  designed  and 
embedded  in  a different  way,  and  we  cannot  assume  that  it  will  take  place  without  a 
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structure,  leader,  definite  timeline,  goals,  objectives,  activities,  assessment,  and 
evaluation,  followed  by  the  continuous  cycle  of  plan,  act,  do,  and  check  (Deming,  1956) 
with  fully  engaged  leadership.  Therefore,  the  time  for  collaboration  must  be  clear  and 
built  into  the  schedule;  the  purpose  and  expected  outcomes  must  be  explicit;  professional 
development  must  be  mandatory  for  participants;  and  professionalism  as  a value  must  be 
exhibited  through  responsible  action  with  accountability  for  results. 

We  are  seeking  a culture  change.  One  that  involves  the  following  transformation: 


Serendipity 

Intentional 

Individual 

Group 

Isolation 

Collaboration 

Disciplinary 

Interdisciplinary 

Autonomous 

-> 

Democratic 

Nebulously  defined  performance 

Criterion  referenced  or  defined  performance 

Results  are  not  defined 

Results  oriented 

Double  Loop  Learning 

Argyris  (1976;  Smith,  2001)  proposed  the  double  loop  learning  theory,  where  the  change 
relates  to  the  change  in  underlying  values  and  assumptions,  lending  its  use  to  more 
complex  and  ill-structured  problem  solving  that  changes  as  the  problem  solving 
advances.  It  is  based  on  a “theory  of  action,”  where  there  is  a distinction  between  an 
individual’s  “espoused  theory”  and  his/her  “theory  in  use”  or  what  they  do.  Double  loop 
learning  brings  the  two,  one’s  “espoused  theory”  and  “theory  in  action,”  together.  In 
other  words,  there  is  a “split  between  theory  and  action”  or  what  we  do  as  practitioners 
and  how  we  speak  of  our  actions  to  others  (theories  in  use).  The  words  we  use  to  explain 
what  we  do  or  to  tell  others  what  we  think  we  do  are  “espoused  theory.”  So  is  there  a 
match  between  intention  and  outcome?  If  there  is  a mismatch  between  the  two,  then  there 
is  conflict.  Usually,  there  would  need  to  be  interaction  with  others  to  discover  a conflict 
between  what  one  espouses  and  what  one  does.  There  are  two  responses  to  the 
mismatches  or  conflicts  between  what  people  say  they  are  doing  and  what  they  actually 
do.  The  first  response  could  be  that  when  something  goes  wrong,  people  will  simply  look 
for  another  strategy  that  will  work  within  the  governing  variables.  An  alternative 
response,  however,  could  result,  which  would  mean  that  rather  than  merely  selecting 
another  strategy,  someone  would  actually  challenge  the  governing  variables  and  try  to 
shift  the  way  in  which  strategies  and  consequences  are  framed;  this  is  double  loop 
learning.  A good  example  occurs 

when  the  error  detected  and  corrected  permits  the  organization  [or  person]  to 
carry  on  its  present  policies  or  achieve  its  present  objectives,  then  that  error-and- 
correction  process  is  single  loop  learning.  Single-loop  learning  is  like  a 
thermostat  that  learns  when  it  is  too  hot  or  too  cold  and  turns  the  heat  on  or  off. 
The  thermostat  can  perform  this  task  because  it  can  receive  information  (the 
temperature  of  the  room)  and  take  corrective  action.  Double  loop  learning  occurs 
when  the  error  is  detected  and  corrected  in  ways  that  involve  the  modifications  of 
an  organization’s  [or  person’s]  underlying  norms,  policies,  and  objectives. 
(Argyris  & Schon,  1974,  pp.  4-30) 
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The  basic  steps  in  this  action  theory  learning  process  are 

1 . discovery  of  espoused  and  theory-in-use 

2.  invention  of  new  meanings 

3.  production  of  new  actions 

4.  generalization  of  results 

The  double-looped  learning  theory  involves  an  individual  in  applying  each  step  to  itself 
(the  theory),  where  underlying  assumptions  or  current  viewpoints  are  questioned  by 
following  through  to  testing  hypotheses  about  behavior  publicly.  This  should  increase 
effectiveness  in  decision-making.  An  improved  acceptance  of  failures  and  mistakes 
should  also  follow  as  a result  of  using  the  four  steps.  The  double  loop  learning  theory  is  a 
“personal  change”  theory  and  is  oriented  toward  professional  education.  A great  example 
for  our  professional  development  of  professors  follows  from  Argyris  (1976): 

A teacher  who  believes  that  she  has  a class  of  ‘stupid’  students  will  communicate 
expectations  such  that  the  children  behave  stupidly.  She  confirms  her  theory  by 
asking  them  questions  and  eliciting  stupid  answers  or  puts  them  in  situations 
where  they  behave  stupidly.  The  theory-in-use  is  self-fulfilling.  Similarly,  a 
manager  who  believes  his  subordinates  are  passive,  dependent  and  require 
authoritarian  guidance  rewards  dependent  and  submissive  behavior.  He  tests  his 
theory  by  posing  challenges  for  employees  and  eliciting  dependent  outcomes.  In 
order  to  break  this  congruency,  the  teacher  or  manager  would  need  to  engage  in 
open  loop  learning  in  which  they  deliberately  disconfirm  their  theory-in-use. 

(p.  16) 

Argysis  (1993)  and  Schon  and  Argyris  (1996)  have  moved  on  further  to  evolve  the 
theory,  focusing  on  the  development  of  action  theory  more  broadly,  “action-science”;  this 
also  impacts  learning  at  the  organizational  level.  The  original  and  more  evolved  theories 
apply  to  our  situation,  as  we  are  striving  to  have  professors  examine  and  challenge  their 
current  values  and  assumptions  within  their  current  schema  (mental  framework  or  map  - 
discussed  within)  while  also  challenging  a broader  learning  by  the  college  as  an 
organization.  The  operational  principles  for  double  loop  learning  are 

1 . Effective  problem-solving  about  interpersonal  or  technical  issues  requires 
frequent  public  testing  of  theories-in-use. 

2.  Double  loop  learning  requires  learning  situations  in  which  participants  can 
examine  and  experiment  with  their  theories  of  action,  (p.  1) 

Argyris  and  Schon  (1974)  present  the  initial  model  with  three  variables  (original  terms): 

• Governing  variables  (Rules):  those  dimensions  that  people  are  trying  to  keep 
within  acceptable  limits.  Any  action  is  likely  to  impact  upon  a number  of  such 
variables;  thus  any  situation  can  trigger  a trade-off  among  governing  variables. 

• Action  strategies  (Behaviours):  the  moves  and  plans  used  by  people  to  keep  their 
governing  values  within  the  acceptable  range. 

• Consequences  (Results):  what  happens  as  a result  of  an  action.  These  can  be  both 
intended  - those  actors  believe  will  result  - and  unintended.  In  addition,  those 
consequences  can  be  for  the  self  and/or  for  others.  (Anderson,  1977) 
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The  work  of  Argyris  and  his  colleagues  set  the  theoretical  stage  for  the  work  of  Senge 
and  others  on  learning,  individually  and  organizationally. 


Figure  A. 6. 9:  Individual  and  Collective  Learning  in  Organizations 


Adapted  from  Swieringa  & Wierdsma  (1992) 


How  People  Learn  was  produced  as  the  final  product  of  a study  to  determine  "how  better 
to  link  the  research  on  the  science  of  learning  to  actual  practice  in  the  classroom"  (The 
National  Research  Council  (NRC),  2000  p.  vii).  The  researchers  focused  on  human 
learning,  learning  research,  and  how  to  help  all  individuals  realize  their  potential. 
Although  this  report  may  often  appear  to  be  K-12  focused,  it  is  equally  focused  and 
meaningful  for  adult  and  professional  development  learning.  The  key  findings,  which  are 
also  supported  by  research,  are 

1 . Students  come  to  the  classroom  with  preconceptions  about  how  the  world 
works.  If  their  initial  understanding  is  not  engaged,  they  may  fail  to  grasp  the 
new  concepts  and  information  that  are  taught,  or  they  may  learn  them  for 
purposes  of  a test  but  revert  to  their  preconceptions  outside  the  classroom. 

2.  To  develop  competence  in  an  area  of  inquiry,  students  must  (a)  have  a deep 
foundation  of  factual  knowledge,  (b)  understand  facts  and  ideas  in  the  context 
of  a conceptual  framework,  and  (c)  organize  knowledge  in  ways  that 
facilitate  retrieval  and  application. 

3.  A metacognitive  approach  to  instruction  can  help  students  learn  to  take 
control  of  their  own  learning  by  defining  learning  goals  and  monitoring  their 
progress  in  achieving  them.  (pp.  14-18) 

The  implications  for  teaching  are  profound: 

• Teachers  must  draw  out  and  work  with  the  preexisting  understandings  that 
students  bring  with  them  - replacing  the  model  of  the  student  as  an  empty 
vessel. . .requiring  the  expansion  of  roles  for  assessment  beyond  traditional  testing. 
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• Teachers  must  teach  some  subject  matter  in  depth,  providing  many  examples  in 
which  the  same  concept  is  at  work  and  providing  a firm  foundation  of  factual 
knowledge,  replacing  superficial  coverage  with  deeper  coverage  [where]  teachers 
have  in-depth  knowledge  of  the  subject  matter  themselves 

• The  teaching  of  metacognitive  skills  should  be  integrated  into  the  curriculum  of  a 
variety  of  subject  areas. . .metacognition  often  teaches  the  form  of  an  internal 
dialogue...  students  may  be  unaware  of  its  importance  unless  teachers  explicitly 
emphasize  it...  [requiring]  the  integration  of  metacognitive  instruction  with 
discipline-based  learning  [and  the  development]  of  strong  metacognitive 
strategies  [embedded  in  the  curriculum].  (NRC,  pp.  19-21) 

Figure  A. 6. 10:  With  Knowledge  of  How  People  Learn,  Teachers  Can  Choose  More 
Purposefully  Among  Techniques  to  Accomplish  Specific  Goals. 


(National  Research  Council,  2000) 
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Therefore,  in  the  design  of  learning  environments: 

• Classrooms  must  be  learner  centered. 

• Attention  must  be  given  to  what  is  taught  (information,  subject  matter),  why  it 
is  taught  (understanding),  and  what  competence  or  mastery  looks  like. 
Learning  with  understanding  is  harder  to  accomplish. . .Many  curricula  present 
too  many  disconnected  facts  in  too  short  a time. . . [Kjnowledge-centered 
environment  provides  [opportunity]  for  depth  of  study. 

• Formative  assessments,  ongoing  assessments  designed  to  make  students' 
thinking  visible  to  both  teachers  and  students  are  essential.  They  permit  the 
teacher  to  grasp  the  students'  preconceptions,  understand  where  the  students 
are  in  the  developmental  corridor  from  informal  to  formal  thinking,  and 
design  instruction  accordingly.  In  the  assessment-centered  classroom 
environment,  formative  assessments  help  both  teachers  and  students  monitor 
progress. 

• Learning  is  influenced  in  fundamental  ways  by  the  context  in  which  it  takes 
place.  A community-centered  approach  requires  the  development  of  norms 
for  the  classroom  and  school  as  well  as  connections  to  the  outside  world  that 
support  core  learning  values ...  [meaning]  norms  such  as  "risk-taking"  or  "don't 
get  caught  not  knowing  something" . . .designing  classrooms  and  activities  in 
ways  that  promote  the  kind  of  intellectual  camaraderie  and  attitudes  towards 
learning  that  build  a sense  of  community. . .establishing  a community  of 
learners.  (NRC  , 2000) 

They  present  a critique  of  both  child,  adult,  and  professional  development  learning 
frameworks.  They 

• are  not  learner  centered 

• are  not  knowledge  centered 

• are  not  assessment  centered 

• are  not  community  centered 

(NRC,  2000,  pp.  23-27) 

In  thinking  further  about  learners  and  learning,  the  NRC  (2000)  distinguished  experts 
from  novices  and  then  how  the  processes  of  learning  develop  expertise  in  a learner.  By 
understanding  expertise,  we  can  better  understand  the  nature  of  thinking  and  problem 
solving.  Expertise  is  not  just  memory  or  intelligence,  not  abilities,  but  rather  an  extensive 
level  of  knowledge  that  affects  everything  about  how  they  learn,  how  they  organize, 
represent,  and  interpret  information  in  their  environment.  This  affects  remembering, 
reasoning,  and  problem  solving.  They  hold  the  following  key  principles  to  be  true: 

1 . Experts  notice  features  and  meaningful  patterns  of  information  that  are  not 
noticed  by  novices. 

2.  Experts  have  acquired  a great  deal  of  content  knowledge  that  is  organized  in 
ways  that  reflect  a deep  understanding  of  their  subject  matter. 

3.  Experts'  knowledge  cannot  be  reduced  to  sets  of  isolated  facts  or  propositions 
but  instead  reflects  contexts  of  applicability:  that  is,  the  knowledge  is 
"conditionalized"  on  a set  of  circumstances. 

4.  Experts  are  able  to  flexibly  retrieve  important  aspects  of  their  knowledge 
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with  little  attentional  effort. 

5.  Though  experts  know  their  disciplines  thoroughly,  this  does  not  guarantee  that 
they  are  able  to  teach  others. 

6.  Experts  have  varying  levels  of  flexibility  in  their  approach  to  new  situations. 

(p.31) 

In  discussing  "meaningful  patterns  of  information,"  the  NRC  (2000)  authors  examine 
some  studies  on  chess  players  (experts)  and  how  they  "chunk  meaningful  information 
together,"  making  it  more  possible  to  extend  short  term  memory  capacity.  Novices  do  not 
have  the  hierarchical,  highly  organized  structure  of  domain  to  use  "chunking"  (DeGroot, 
1965).  Other  types  of  experts  have  similar  skills,  for  example,  experts  in  electronic 
circuitry  and  computer  programming  (Egan  & Schwartz,  1979;  Ehrlich  & Soloway, 

1984).  Electronics  technicians  can 

reproduce  large  portions  of  complex  circuit  diagrams  after  only  a few  seconds  of 
viewing;  novices  could  not.  The  expert  circuit  technicians  chunked  several 
individual  circuit  elements  (e.g.  resistors  and  capacitors)  that  performed  the 
function  of  an  amplified.  By  remembering  the  structure  and  function  of  a typical 
amplifier,  experts  were  able  to  recall  the  arrangement  of  many  of  the  individual 
circuit  elements  comprising  the  amplifier  chunk,  (p.  33) 

Mathematics  experts,  physicists,  and  expert  teachers  have  developed  these  expert 
"schemas"  or  organized  conceptual  frameworks  that  guide  how  one  processes 
information  (Glaser  & Chi,  1988;  Hinsley,  Hayes,  and  Simon,  1977;  Robinson  & Hayes, 
1978;  Scarborough,  1984).  Schemata,  the  frameworks,  and  "chunking,"  a process,  are 
capabilities  developed  by  experts  over  time.  If  we  were  to  dig  more  deeply  into  the  field 
and  subfields  of  the  information  process,  we  would  get  into  breaking  out  the  units 
individuals  are  capable  of  processing,  coupled  with  how  they  learn  something:  the 
resulting  outcomes.  Eor  example,  the  Scarborough  study  engaged  the  researcher  in  the 
breakdown  of  all  information  to  be  learned  into  memory  units  as  part  of  the  process  of 
preparing  the  learning  materials  for  the  experiments.  When  considering  the  organization 
of  knowledge,  experts'  knowledge  is  "organized  around  core  concepts  or  'big  ideas'  that 
guide  their  thinking  about  their  domain"  (NRC,  2000,  p.  36).  This  affects  how  they 
understand  and  represent  problems.  The  NRC  report  thoroughly  discusses  this  and 
provides  examples. 
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Figure  A. 6. 11:  Network  Representations  of  Incline  Plane  Schema  of  Novices  and  Experts 
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Another  difference  between  experts  and  novices  is  one  of  accessing  knowledge  or 
retrieving  what  is  relevant  to  a particular  task.  Their  knowledge  is  "contextualized  - 
includes  a specification  of  the  contexts  in  which  it  is  useful"  (Glaser,  1992,  pp.  42-43; 
Miller,  1956;  Simon,  1980).  This  concept  has  important  implications  for  teaching, 
assessment,  and  curriculum  development.  It  is  important  that  all  three  help  students  begin 
to  "contextualize"  knowledge  as  they  learn  it.  When  considering  assessment  in  light  of 
"contextualization,"  it  is  important  to  understand  that  many  tests  fail  to  engage  students  in 
determining  when  knowledge  is  to  be  useful.  For  example,  when  mathematics  students 
are  "asked  whether  the  formula  that  quantifies  the  relationship  between  mass  and  energy 
is  E=MC,  E=MC^,  or  E=MC^,  a correct  answer  requires  no  knowledge  of  the  conditions 
under  which  it  is  appropriate  to  use  the  formula"  (Glaser,  p.  43).  Therefore,  "fluent 
retrieval"  involves  a number  of  sub  processes  that  vary  from  fluent  to  automatic.  This 
does  not  mean  that  experts  access  or  process  faster  because  they  approach  problems  more 
deeply,  trying  to  understand  it  and  thus  often  take  more  time.  Developing  fluency  is 
important  because  in  doing  so,  there  are  fewer  demands  requiring  conscious  attention. 

The  easier  the  processing,  the  greater  one's  capacity  to  tend  to  other  aspects  of  the  task 
(Anderson,  1981,  1982;  EaBerge  & Samuels,  1974;  Eesgold  et  ah,  1988;  Miller,  1956; 
Schneider  & Shiffrin,  1985).  A good  example  is  driving  a car.  Eor  the  most  part,  expert 
drivers  can  multitask  while  driving,  without  getting  into  a wreck.  Experience  builds  that 
fluency  to  assess  information  subconsciously  while  attending  to  other  tasks  consciously. 
Analogously,  novice  readers  who  are  less  fluent  in  decoding  works  cannot  focus  on 
understanding  what  they  are  reading  because  they  are  inexperienced  in  reading.  Those 
who  read  well  (also  keyboard  well)  do  not  process  in  single  words;  they  process  in 
phrases,  then  sentences,  and  then  paragraphs  as  they  become  more  expert.  Speed  readers 
may  not  remember  the  exact  wording;  they  read  only  for  meaning  or  essence. 

Adaptive  expertise  is  highly  critical.  This  means  that  some  ways  of  organizing 
knowledge  help  people  to  be  more  flexible  and  adaptable  to  new  situations  requiring  the 
use  of  knowledge.  Experts  can  be  skilled,  or  beyond  that  to  highly  competent,  because 
they  can  adapt  knowledge  more  readily  to  meet  external  or  changing  demands:  a cook 
versus  a chef  who  can  improvise  and  create  when  surprised  by  an  unexpected  request  or 
an  artist  versus  a virtuoso  who  creates  while  entertaining.  Adaptive  expertise  is  not 
bounded;  in  other  words,  one  can  use  it  in  unexpected  and  unplanned  ways.  When 
designing  information  systems,  the  designer  could  consider  him/herself  "bound"  by  the 
technical  specifications  described  by  the  customer  or  he/she  could  consider  those 
specifications  the  lowest  level  of  performance  and  as  a point  of  departure  or  for  further 
exploration. 

Adaptive  expertise  is  really  the  ability  we  want  to  develop  in  our  students  and  in 
ourselves  because  this  ability  creates  the  continuous  lifelong  learner  who  is  never 
bounded  by  moment,  job,  assignment,  or  company.  "Adaptive  learners  are  able  to 
approach  new  situations  flexibly  and  to  learn  while  engaged  in  something  new"  (Glaser, 
1992,  p.  47).  A very  good  example  of  that  is  when  someone  is  most  satisfied  being  given 
an  assignment  for  which  they  do  not  really  know  everything  needed  to  accomplish  the 
assignment.  Adaptive  learners  are  happiest  in  this  type  of  scenario,  whether  in  school  or 
at  work.  Adaptive  learners  use  what  they  have  learned  and  learn  while  engaged;  they 
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operative  metacognitively,  continuously  challenging  or  seeking  higher  levels  of  expertise 
- more  often  than  not,  subconsciously.  They  attempt  to  move  beyond  where  they  are  all 
the  time.  They  do  NOT  simply  do  the  same  things  more  efficiently  (in  this  they  become 
very  quickly  bored,  unmotivated,  and  unsatisfied);  instead  they  attempt  to  understand 
things  more  deeply,  do  things  better  or  differently,  challenge  the  status  quo,  and  seek 
change.  These  are  our  virtuosos,  whether  faculty  members  or  our  students.  This  is  our 
goal! 

Implications  for  teaching  - it  is  not  possible  to  assume  that  because  someone  is  an  expert 
in  a discipline  they  can  then  teach  it.  They  often  forget  what  is  difficult,  thus  their 
instruction  becomes  ineffective.  That  is  why  it  benefits  us  to  work  in  interdisciplinary 
groups.  Also  disciplinary  knowledge  is  not  the  same  as  pedagogical  knowledge.  We 
understand  that  in  this  initiative;  thus  our  faculty  members  are  exploring  major  content 
areas  related  to  teaching  and  learning:  course  analysis,  test  analysis,  development  of 
student  learning  outcomes,  assessment  planning,  test  development,  performance 
assessment  development,  teaching  models,  educational  research,  and  a myriad  of 
subtopics.  They  will  then  design  experimental  pilots  to  guide  classroom  research  with 
students  on  learning. 

The  NRC  (2000)  report  goes  further  to  discuss  other  aspects  of  learning  with  direct 
implications  for  teaching.  Transfer  of  learning  is  essential  to  understanding  the 
development  of  competencies.  Transfer  of  learning  is  "the  ability  to  extend  what  has  been 
learned  in  one  context  to  new  contexts”  (Byrnes,  1996,  p.  74).  The  results  are  different 
when  testing  for  learning  versus  testing  requiring  transfer  of  knowledge.  Research  on  this 
has  changed  in  focus;  it  retains  the  "practice"  component  but  works  toward  understanding 
the  difference  in  results  across  kinds  of  practice.  Today's  research  on  transfer  also 
considers  learner  characteristics  (e.g.  existing  knowledge  and  strategies).  Key 
characteristics  to  consider  about  learning  and  transfer  are 

• Initial  learning  is  necessary  for  transfer,  and  a considerable  amount  is  known 
about  the  kinds  of  learning  experiences  that  support  transfer. 

• Knowledge  that  is  overly  contextualized  can  reduce  transfer;  abstract 
representations  of  knowledge  can  help  promote  transfer. 

• Transfer  is  best  viewed  as  an  active,  dynamic  process  rather  than  a passive 
end-product  of  a particular  set  of  learning  experiences 

• All  new  learning  involves  transfer  based  on  previous  learning,  and  this  fact 
has  important  implications  for  the  design  of  instruction  that  helps  students 
learn,  (p.  53) 

Elements  that  promote  initial  learning  relate  to  the  degree  of  mastery  of  the  original 
subject;  otherwise  there  is  not  enough  of  a basis  for  transfer.  Transfer  can  be  affected  by 
the  degree  people  memorize  versus  understand  and  how  much  time  they  are  given  to 
learn,  especially  if  a complex  subject  matter.  The  amount  of  time  provided  must  be 
directly  proportional  to  the  complexity  of  the  material.  If  the  meanings  are  not  initially 
clear,  it  will  take  learners  time  to  process  and  reach  the  point  of  some  understanding. 
Moving  too  quickly  with  too  many  concepts  will  inhibit  learning  and  understanding,  as 
they  will  learn  only  isolated  facts  rather  than  creating  organizing  principles  that  make 
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meaning  and  bring  about  understanding  of  learning.  But  they  need  enough  time  to 
process  new  or  expanded  knowledge.  Deliberate  practice  is  encouraged,  especially  to 
include  monitoring  one's  own  learning  experiences  where  feedback  is  sought  and  used  - 
critical  to  successful  learning  (Chi,  Bassok,  Lewis,  Reimann,  & Glaser,  1989,  1994; 
Ericsson,  Krampe,  & Tesch-Romer,  1993;  Thorndike,  1913).  One  method  that  is 
successful  in  deepening  understanding  is  to  "contrast"  when,  where,  and  why  to  use  new 
knowledge. 

This  returns  us  to  the  "contextualization"  of  knowledge  thoughts  presented  above. 
Transfer  is  learned  better  if  students  can  see  potential  transfer  implications  related  to 
what  they  are  learning.  Therefore,  these  contrast  opportunities  can  be  beneficial  toward 
that  end  as  well.  "Competence  motivation  is  the  amount  of  time  people  are  willing  to 
devote  to  learning"  (NRC,  2000,  p.  60).  Most  human  beings  are  motivated  to  develop 
competence  and  to  solve  problems,  working  hard  for  intrinsically  rewarding  reasons.  The 
level  of  difficulty  can  be  defined  by  the  challenges  presented.  There  must  be  a match  of 
difficulty  to  ability  to  maintain  motivation  because  if  tasks  are  too  difficult  or  too  easy, 
learners  lose  interest  because  of  boredom  or  frustration.  If  learners  who  are  performance 
oriented  are  concerned  about  errors,  those  learning-oriented  like  new  challenges  (Dweck, 
1989).  Learners  are  motivated  by  social  opportunities  to  learn  because  of  feeling  like  they 
are  contributing  to  others  (Schwartz,  Sin,  Brophy,  & Bransford,  1999).  If  learners  see 
usefulness  in  what  they  are  learning  and  that  they  can  use  what  is  learned  to  impact 
others,  they  are  also  more  motivated  (McCombs,  1996;  Pintrich  & Schunk,  1996; 
Vanderbilt,  1998).  Other  factors  affecting  transfer  can  be  the  a)  the  learning  context  of 
the  original  learning  and  b)  representation  of  problems  at  higher  abstract  levels,  meaning 
that  if  problems  are  presented  so  specifically,  students  will  want  to  transfer  their  solutions 
inappropriately.  It  helps  students  to  generalize  their  solutions.  Students  need  to 
understand  the  underlying  principles,  and  a "suite  of  representations"  enables  them  to 
more  flexibly  transfer  (Spiro,  Feltovich,  Jackson,  & Coulson,  1991). 

Transfer  can  be  perceived  as  a relationship  between  learning  and  transfer  conditions:  "the 
amount  of  transfer  will  be  a function  of  the  overlap  between  the  original  domain  of 
learning  and  the  novel  one"  (NRC,  2000,  p.  63).  They  also  argue  that  "transfer  is  a 
function  of  the  degree  to  which  the  tasks  share  cognitive  elements"  (NRC,  p.65; 
Thorndike  & Woodworth,  1901;  Woodworth,  1938).  Often  in  technical  fields,  we  limit 
learning  about  theory  or  contain  the  process  of  abstractions;  Biederman  and  Shiffrar 
(1987)  and  Anderson,  Reder,  and  Simon  (1996)  found  benefits  from  providing  students 
20  minutes  on  abstract  principles  before  engaging  them  in  the  task  to  increase  learning. 
There  is  more  research  providing  strong  support  for  learning  benefits  by  helping  students 
represent  their  experiences  at  more  abstract  levels  because  it  helps  them  transcend  the 
specificity  of  particular  contextualized  learning  or  examples  (Klahr  & Carver,  1988; 

NRC,  1994;  Singley  & Anderson,  1989).  The  NRC  report  goes  further  to  discuss  active 
versus  passive  approaches  to  transfer  and  to  transfer  and  metacognition,  meaning  that 
students  become  more  aware  of  themselves  as  learners  and  monitor  their  own  growth  and 
learning  strategies,  resources  and  become  able  to  assess  their  readiness  for  tests  and 
performances.  Metacognitive  approaches  to  learning  seem  to  positively  affect  the  degree 
to  which  students  will  transfer  to  new  situations  without  explicit  prompting  (active  / 
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passive  transfer).  One  of  these  methods  is  "reciprocal  teaching,"  where  teachers  and 
students  take  turns  leading  the  learning  (NRC,  p.  67). 

It  might  be  important  to  consider  learning  as  transfer  from  previous  experiences.  Very 
often  people  think  of  transfer  as  learning  something  and  then  applying  it  somewhere  else. 
However,  the  NRC  (2000)  report  establishes  that  even  initial  learning  requires  transfer 
because  people  bring  knowledge  to  any  learning  situation,  even  initial  ones.  Thus  "all 
learning  involves  transfer  from  previous  experiences"  (p.  68).  The  implications  are  that 
students  may  have  inactivated  knowledge  relevant  to  what  is  about  to  be  learned,  which 
when  activated  can  build  on  student  strengths.  Previous  knowledge  can  lead  students  to 
misinterpretation  of  new  knowledge  when  they  begin  to  try  to  construct  understandings 
or  meanings.  One  I find  very  interesting  and  have  dealt  with  before  is  that  students  may 
find  the  teaching  practices  conflict  with  those  they  have  experienced  before.  For  example, 
NIU  Mathematics  requires,  or  did,  manual  calculation  of  equations  on  their  mass  tests,  no 
technology-calculators-allowed.  This  has  been  hypothesized  to  distract  students  coming 
from  other  learning  environments  who  were  allowed  to  use  technology,  causing  them  to 
perform  less  well.  NIU's  argument  is  that  they  can  better  assess  student  understanding; 
however,  one  feeder  institution  changed  its  tests  and  improved  items  so  that  measurement 
of  understanding  could  be  attained  using  technology.  So  building  on  existing  knowledge 
can  be  beneficial  or  can  cause  difficulties  in  learning.  There  can  also  be  a conceptual 
change  in  knowledge.  "Teaching  by  telling,"  direct  instruction,  or  lectures  can  be 
dangerous  because  when  learners  construct  new  knowledge  or  meanings  based  upon 
current  knowledge  and  conditions  are  unclear  because  of  lack  of  conditions,  broader 
examples,  contrasts,  situations,  they  may  not  be  able  to  continue  to  build  knowledge 
accurately  (p.  71).  The  role  of  culture  needs  to  be  considered.  If  there  is  a mismatch 
between  home  and  learning  environments,  difficulties  may  arise,  which  seems  to  me  to 
be  more  of  an  issue  in  K-12  than  in  higher  education. 

The  design  of  learning  environments  has  multiple  important  considerations.  They  should 
be  integrated  and  learner  centered,  knowledge  centered,  and/or  assessment  centered  with 
formative  assessments  using  feedback,  and  then  ultimately  have  a summative  assessment 
or  evaluation  (NRC,  2000). 


Figure  A. 6. 12:  Perspectives  On  Learning  Environments 


(Bransford  et  ah,  1998  as  cited  in  NRC,  2000,  p.  143) 
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Transforming  Knowledge 


A learning  community  is  one  where  constructivism,  reflective  practice,  and  collaboration 
are  alive  and  functioning  well.  Knowledge  is  constructed.  Knowledge,  application,  and 
learning  are  deepened  through  reflection,  and  as  established  above,  learning  is  enhanced 
through  the  social  process  of  collaboration  (Sullivan  & Glanz,  2006).  Scarborough  (2001) 
found  when  interviewing  manufacturing  leaders  that  reflection  was  a necessary, 
rewarding  and  productive  component  of  their  daily  process  as  leaders;  in  fact,  they 
realized  they  must  have  a "quiet  time"  each  day  to  process  what  had  occurred  and 
generate  a response  or  the  next  step.  Therefore,  reflection  is  not  just  a phenomenon 
within  the  educational  or  teaching  context,  it  is  truly  a requirement  of  effective  leaders  or 
performers  in  other  sectors.  Learning  communities  are  not  unique  to  education;  they  do  or 
should  exist  in  every  type  of  organization  as  a means  to  accomplish  ongoing,  effective, 
purposeful,  meaningful,  and  continuous  change. 

In  beginning  our  thoughts  about  knowledge,  Stacey  (2001)  defines  it  as 

a process,  and  a relational  one  at  that,  which  cannot  therefore  be  located  simply  in 
an  individual  head  to  be  extracted  and  shared  as  an  organizational  asset. 
Knowledge  is  the  act  of  conversing,  and  learning  occurs  when  ways  of  talking, 
and  therefore  patterns  of  relationship  change. . .The  knowledge  assets  of  an 
organization  lie  in  the  pattern  of  relationships  between  its  members,  (p.  98) 

Thomas  et  al.  (2005)  focus  mostly  on  Gardner's  work  described  below.  But  they  do  make 
some  important  observations.  Although  we  may  have  learning  style  preferences  as 
learners,  life  requires  us  to  use  multiple  styles.  Solving  problems,  for  example,  requires 
us  to  recall  information,  but  it  also  may  require  us  to  take  some  action  using  our  bodies  or 
oral  linguistic  skills,  etc.  Experiential  learning  (to  be  discussed  more  thoroughly  in 
pedagogy)  combines  theory  with  experience,  mirroring  reality,  solving  problems  through 
the  application  of  knowledge,  beginning  with  the  present  and  moving  one  forward  to  the 
future.  Projects,  if  designed  accordingly,  provide  the  opportunity  for  learners  to  discover 
knowledge  or  construct  it  for  themselves,  requiring  individual  and  collective  thought  and 
problem  solving.  Traditional  education,  and  this  includes  professional  development  for 
faculty,  often  requires  listening,  memorizing,  and  then  repeating  what  can  be  recalled. 
That  process  starts  in  the  present  and  moves  learners,  whether  students  or  faculty, 
backward  to  the  past,  requiring  them  to  remember  what  others  have  already  discovered 
rather  than  discovering  through  projects  that  engage  them  in  the  construction  of 
knowledge.  In  this  scenario,  we  provide  learners  with  the  answers  rather  than  questions 
and  problems,  indicating  that  the  more  one  can  remember,  the  more  one  knows.  This 
does  not  make  problem  solving  a theory  in  learning.  In  society,  however,  learners  are 
required  to  actively  plan,  observe,  test,  and  reflect  rather  than  the  other  more  passive 
approach.  Teachers  (professors)  might  consider  this  question,  "how  can  one  learn  to  be  an 
effective  teacher  without  actually  teaching?"  (p.  78).  This  means  that  teaching  is  defined 
differently  than  "imparting  [or  dumping]  information  into  repositories”  students’  heads, 
where  the  burden  of  learning  is  on  the  teacher.  Active  or  inquiry-based  learning  puts  the 
burden  equally  on  the  student,  requiring  them  to  explore  the  unknown.  One  is  static:  the 
model  above  where  recall  is  all  that  is  required;  however,  knowledge  is  dynamic,  always 
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changing  because  we  begin  with  the  present  and  move  towards  the  future.  This  proeess 
requires  problems,  problem  solving,  and  self-direeted  and  eollaborative  learning. 

For  adult  learning  (the  learners  in  this  initiative  or  eollege  age  students),  Wald  and 
Castleberry  (2000)  identify  five  assumptions: 

inquir[ing]  into  underlying  assumptions  deepens  the  learning  process;  learning 
is  an  aetive  proeess  that  oeeurs  over  time;  learning  is  driven  by  the  learner 
around  meaningful  issues;  learning  is  experiential  by  nature;  and  learning  is 
fueled  by  rieh,  diverse,  aeeessible  sourees  of  information,  (p.  110) 

Although  how  learning  best  oeeurs  is  to  be  thoroughly  diseussed  later,  it  is  important  to 
weave  thoughts  on  it  into  other  seetions.  We  have  required  our  partieipants  to  engage  in 
aetive  learning  around  meaningful  issues  and  for  authentie  purpose.  Minnieh's  (2005) 
book  is  extremely  enlightening  when  eonsidering  knowledge  and  its  meanings.  Although 
it  is  fairly  impossible  to  simplify  and  distill  its  eontents  while  holding  to  her  eomplex,  yet 
noteworthy,  presentation  about  the  topie  (transforming  knowledge),  I will  try  to  at  least 
stimulate  readers  to  get  the  book  and  read  it  on  their  own.  Minnieh  presents  the 
transformation  of  knowledge  from  a soeialized  eontext,  bringing  to  this  projeet  a broader 
viewpoint  from  the  perspeetive  of  gender,  elass,  and  race.  Her  explanations  are  eritieal 
when  working  with  multieultural  partners  in  the  aeademy  and  equally  important  for 
eonsidering  how  best  to  aehieve  learning  and  making  meaning  when  teaehing  a 
multieultural  student  body.  In  her  diseussion  on  rationality,  intelligence,  and  good  papers, 
she  realized  when  performing  a elassroom  experiment  that  the  group  that  seemed  to  be 
more  intelligent  in  writing  their  papers  had  heard  her  reeognize  that  "the  aeademie  paper 
was  only  one  form  or  way  of  aehieving,  expressing,  and  eommunieating  fully  'rational' 
understanding"  (p.  182).  She  reeognized  that  each  student  is  eapable,  "when  working 
from  her  or  his  own  strengths,  of  exeellent  rational  work.  We  pay  a high  priee 
eolleetively  as  well  as  individually  for  'training'  so  many  wonderful  minds  into  too  few 
modes”  of  performanee  options  (p.  183).  She  goes  further  to  aeknowledge  the  wide  range 
of  modes  her  "thinking"  friends  and  students  eross:  "analytieal,  synthetie,  faetual, 
imaginative,  holistie,  atomistie,  relational,  argumentative,  suggestive,  symbolie, 
juxtapositional,  oppositional,  deduetive,  indieative,  narrative,  poetie,  eonerete,  intuitive, 
abstraet,  totalizing,  particularizing,  [and]  dramatic"  (p.  183).  When  we  define  rational  as 
only  one  mode,  we  exelude  all  the  other  modes  and  all  those  who  rationalize  (learn 
knowledge  and  make  meaning  of  that  knowledge)  using  those  modes. 

Minnieh  (2005)  eonsiders  transforming  knowledge  from  the  perspeetive  of  inclusion, 
across  races  and  genders,  and  diseusses  who  may  be  exeluded  if  opportunity  is  limited  or 
not  available  beeause  of  who  someone  is.  She  is  very  eoneerned  with  "dominant  meaning 
systems,"  where  the  dominant  eulture  results  in  partieular  groups  being  shut  out  or 
exeluded  and  where  there  are  distortions  that  oecur  because  of  exelusion.  She  diseems  the 
importance  of  eurrieula  and  its  role  in  working  to  understand  the  new  rather  than  a 
eontinuous  unfolding  of  all  that  has  already  been  established  and  notes  that  we  must 
remove  sueh  distortions  - thus  "that  whieh  is  aetively  exeluded  from  the  eurrieulum  is 
very  likely  to  be  forgotten,  seen  as  deviant  and  marginal"  (p.  59).  Making  meaning  must 
be  open  to  everyone,  not  dominated  by  any  one  perspective  or  definition,  where  the  past 
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and  future  are  bridged,  so  new  generations  will  make  more  relevant  meaning.  She  rejects 
"mainstreaming"  because  it  implies  that  there  is  one  main  stream  (that  normal  is  the  goal) 
and  uses  a metaphor  of  becoming  invisible  in  a big  river.  One  of  her  most  significant 
points  is  that  "transformation,  in  contrast,  places  the  emphasis  not  on  joining  what  is,  or 
on  adding  something  onto  it,  but  on  changing  it"  (p.  60).  She  warns  against  circular 
reasoning,  where  "circularity  extends  the  error  of  generalizing  too  far  from  too  few  into 
the  standards  by  which  the  hierarchy  is  maintained  to  such  an  extent  that  the  few 
reappear,  tellingly,  as  the  'ideal'"  (p.  156).  Prejudice  is  one  kind  of  circular  reasoning. 

This  means  that  established  judgments  are  not  reconsidered  in  light  of  new  experience  or 
evidence  that  changes  those  judgments.  Our  students  are  also  prejudiced  in  that  sense. 
They  feel  that  any  group  excluded  was  rightfully  excluded.  "A  citizenry  that  is 
humanistically  aware  is  a citizenry  that  is  capable  of  confronting  diversity,  ambiguity, 
and  conflict,  overcoming  prejudice  and  self-interest,  enlarging  its  sympathies,  tackling 
tough  public  issues,  and  envisioning  possibilities  beyond  the  limits  of  circumstance"  (The 
Humanities  and  the  American  Promise,  the  National  Endowment  for  the  Humanities-A 
Report,  as  cited  in  Minnich,  p.  166).  Rather  than  considering  our  curricula  as  a set  of 
academic  disciplines,  it  is  more  enlightened  to  consider  them  as  "certain  ways  of 
thinking  - inquiring,  evaluating,  judging,  finding,  and  articulating  meaning"  (p.  167). 
Minnich  discusses  many  of  our  social  situations  to  illuminate  thought  on  how 
transforming  knowledge  can  be  dominated  through  exclusionary  practices. 

Transforming  knowledge,  simply,  is  to  create  meanings.  Humans  learn  through  creating 
meanings,  which  can  vary  across  groups.  But  we  are  "conscious  creatures  and  creators  of 
meaning"  (Minnich,  2005,  p.  xxi).  Very  few  words  are  neutral;  they  are  usually 
personally,  intellectually,  historically,  culturally,  politically,  and/or  often  racially 
charged.  Words  reflect  knowledge  and  meanings  of  the  times  and  the  situations  in  which 
they  come  into  use.  Words  have  multiple,  sometimes  contradictory,  meanings  over  time, 
especially  across  cultures  and  subcultures.  When  considering  meanings, 

one  is  trying  to  comprehend  a (re)  framing  of  available  meanings,  a task  that 
requires  attention  to  each  word,  each  line,  each  section  within  the  context  of  the 
entire  work,  itself  read  within  its  own  multiple  contexts.  In  this  process  of 
reading,  listening,  opening  to  take  in  "what  is  going  on  here,"  philosophical 
readers,  like  effective  political  actors,  attentive  parents,  good  teachers,  artful 
psychological  and  pastoral  counselors,  listen  for  how  what  is  said  coheres,  and 
[what]  does  not;  how  it  is  familiar  and  strange;  [and]  how  it  invokes  and  suggests, 
and  suppresses  things  not  directly  said.  They  listen  for  recurring  images  and  for 
what  sorts  of  relations  those  images  privilege  (mechanical?  organic?  rigid?  fluid? 
oppositional?  transactional?).  They  pick  up  on  language  use  and  what  it  suggests: 
why  the  colloquialism  here,  the  technical  term  there?  Why  that  rhythm  in  those 
sentences,  another  in  these? 

We  all  do  all  of  that  - and  more,  of  which  we  are  rarely  aware  - to  be  fully 
present  within  a conversation  as  it  is  happening  here  and  now,  with  these 
particular  people.  We  can  do  that  because  and  insofar  as  we  also  always  hold  in 
mind  other,  differing  conversations.  Making  sense  with  one  another,  which  is 
both  enabled  and  limited  by  culturally  framed  interactions,  is  an  ongoing  project 
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that  can  never  be  completed.  It  is  a transactional  process  that  has  no  products  but 
does  have  crucial  effects.  And  those  effects  are  of  great  importance:  through  them 
we  can  be  deformed  as  well  as  informed,  and  sometimes  even  transformed. 

There  is  nothing  trivial  or  'only'  theoretical  about  how  people  in  their  daily  lives 
make  sense  together,  and  this  ongoing  process  is  particularly  significant  when  we 
are  trying  to  make  sense  of  who  or  what  "we"  are,  of  who  and  what  "they"  are.  In 
doing  so,  we  are  making  and  remaking  our  lives  and  possible  relations  with  others 
and  with  the  earth  we  share.  So  calling  on  every  art  of  listening. . . , and  trying 
always  to  practice  it  better,  ...  to  locate  where  and  how  efforts  to  connect  with 
others  are  distorted  into  prejudicial— preformed,  unreflective,  and  so  potentially 
dangerous  - forms . . . .The  danger  lies  in  mis-taking  what  is  before  us  by  forcing  it 
into  frames  of  meaning  within  which  it  cannot  reveal  its  unruly  uniqueness. 

(pp.  4-5) 

The  history  of  thought  [in  contrast  to  the  history  of  ideas]  is  the  analysis  of  the 
way  an  unproblematic  field  of  experience,  or  a set  of  practices,  which  were 
accepted  without  question,  which  were  familiar  and  "silent,"  out  of  discussion, 
becomes  a problem,  raises  discussion  and  debate,  incites  new  reactions,  and 
induces  a crises  in  the  previously  silent  behavior,  habits,  practices,  and 
institutions.  The  history  of  thought,  understood  in  this  way,  is  the  history  of  the 
way  people  begin  to  take  care  of  something,  of  the  way  they  become  anxious 
about  this  or  that.  (Foucault,  2001,  p.  74  as  cited  in  Minnich,  2005,  p.  5) 


Foucault... calls  us  into  recurrently  fresh  thinking,  instead  of  being  adopted  as  yet 
another  method  to  be  learned  and  applied.  (Minnich,  2005,  p.  5) 

Whether  we  are  discussing  the  transformation  of  knowledge  into  useful  meanings  by  our 
students  or  the  transformation  of  knowledge  by  ourselves,  professors  in  higher  education, 
into  useful  meanings  by  the  study  of  something,  a challenge  occurs,  one  of  which  (for  us) 
is  "How  do  thinking  and  knowledge  relate  to  the  'real  world'?"  Personally,  as  much  as  I 
agree  with  some  of  the  thought  presented  above  about  the  academy  being  a separate  place 
where  thought  and  study  should  rein  free  safely,  I also  believe  that  when  we,  professors, 
are  perceived  in  that  manner  and  not  considered  part  of  the  "real  world"  that  implies  we 
are  not  "working."  For  those  of  us  who  are  dedicated  to  our  fields,  students,  service,  and 
to  using  our  knowledge  and  study  to  evolve  solutions  or  new  directions,  we  are  part  of 
that  real  world.  Especially  when  we  are  striving  to  extend  students'  minds  and  capacities, 
we  are  very  much  part  of  that  real  world,  often  struggling  with  how  to  challenge  and  lead 
students  to  achieve  their  highest  potential  with  us,  constructing  knowledge  about  how  to 
do  just  that  - increase  student  learning.  Over  the  past  30  years,  higher  education  has 
learned,  changed,  and  learned,  and  it  continues  to  do  so.  There  have  been  new  initiatives, 
new  curricula,  new  institutions  within  the  academy  (e.g..  Center  for  Intercultural 
Relations,  Women's  Center,  Center  for  Study  of  Social  Change,  and  many  more). 
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Finally,  there  are  those  professors  who  still  teach  in  lecture  halls  where  the  seats  are 
bolted  down  in  neat  lines,  with  a lectern  and  screen,  etc.  However,  there  are  also  those 
professors  who  have  created  a different  learning  environment  where  students  sit  around 
round  tables  or  in  study  rooms  informally;  there  are  distance  courses  where  students  can 
interact  regardless  of  where  they  are  (or  at  least  when  the  technology  is  working  and 
professors  have  designed  their  courses  for  technological  interaction).  There  is  a 
movement  for  engaged  learning,  active  learning,  and/or  student/faculty  learning 
communities.  This  is  what  we  are  about.  We  are  going  to  transform  knowledge  into  new 
meanings  for  us  individually  and  collectively  as  teaching  professors,  with  the  intent  of 
changing  how  our  students  transform  knowledge  in  our  courses.  Students  have  changed, 
but  often  we  have  not.  The  best  of  us  has  changed,  is  willing  to  be  unsure  about  meaning, 
and  has  the  capacity  to  build  meaning  with  students  rather  than  for  students.  Otherwise, 
we  become  "a  professor  [who]  is  no  more  competent  than  memory  banks  in  transmitting 
established  knowledge"  (Lyotard,  1988,  p.  53). 

To  save  the  day,  new  ways  of  thinking  are  emerging  from  the  academy.  Some  of  the 
shifts  are  from  singular  to  plural,  "students"  rather  than  "the  student"  because  singular 
terms  often  hide  complexities,  while  plural  terms  help  us  to  acknowledge  differences, 
make  relevance,  and  move  towards  achieving  transcendent  unity.  Plural  is  more 
inclusive.  Another  shift  is  from  nouns  to  verbs,  implying  a more  active  position  away 
from  static,  which  possibly  helps  us  to  focus  on  complex  and  interrelating  systems  rather 
than  on  static  products  and  abstractions.  Changing  language  reflects  a shift  from  asking 
what  "knowledge  is"  to  asking  how  something  "becomes  knowledge"  and  whether  it 
makes  a difference.  For  example,  Focault  (2001)  did  not  think  about  "problems,"  he 
thought  about  "problematizing,"  changing  the  verb  from  static  to  dynamic  or  active, 
which  leads  to  another  shift  where  singular  and  static,  related  to  externally,  are  defined  by 
what  they  are  not,  as  if  they  could  not  be  something  else  - fixed. 

A third  shift  is  one  of  relationalities  where  relationalists  change  from  external  or 
"additive"  to  internal  or  "transactional":  when  being  Black  differs  for  men  and  women; 
being  Chinese  differs  from  being  Chinese  American.  "These  identities  are  imposed, 
internalized,  and  more  or  less  freely  chosen;  they  are  woven  of  many  strands,  sustained, 
and  challenged  through  many  sorts  of  daily  interactions.  They  are  no  'kinds'  that  we 
simply  'are',  one  by  one,  additively,  nor  do  they  intersect  only  at  some  points"  (Minnich, 
2005,  p.  12).  Transactional  relations  focus  on  relations,  not  static  things,  that  enable  one 
to  visualize  the  practical,  applied,  and  political  side  of  theories  or  abstractions.  These 
have  embedded  past  practices  that,  when  applied,  could  confirm  and  continue  its 
meaning,  or  we  can  realize  that  generalizations  or  historical  implications  came  out  of 
particular  social  realities  and  be  careful  not  to  use  them  when  inappropriate.  We  are  to 
continue  the  process  of  expanding,  refining,  and  creating  new  meanings  of  those  theories 
and  practices.  This  leads  us  from  divided  to  mutually  formative  theory  and  practice,  yet 
another  shift.  This  is  the  new  scholarship  in  the  academy.  We  are  to  make  new 
meanings  that  then  translate  into  providing  new  approaches  or  opportunities  for 
our  students  to  make  new  or  to  extend  and  deepen  meanings.  A good  example  of 
these  transitions  is  that  our  word  processors  do  not  yet  reflect  the  shifts  or  their 
vocahularies  (e.g.,  "narrativizing,  significations,  pluralize")  that  move  from  static  to 
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active.  This  can  represent  our  own  resistance  to  new  directions.  On  the  other  hand,  some 
of  us  value  new  language  because  it  represents  new  directions,  thoughts,  and  meanings. 
Living  languages  should  be  limber,  although  at  times  they  may  need  to  be  realigned  to 
correct  that  locked  in;  we  do  not  want  to  substitute  an  old  rigid  for  a new  rigid  posture. 
These  changes  are  to  stimulate  our  minds,  to  enable  us  to  think  more  openly  and  flexibly, 
and  to  enable  us  to  be  more  responsive,  as  opposed  to  reactive. 

Minnich  (2005)  goes  further  to  establish  that  critiquing  curricula  is  critical  because  of  the 
framework  of  meaning  behind  particular  questions  of  what  to  teach  and  to  whom. 
Curricula  engage  us  in  knowledge  making  and  participation  in  understanding.  In  a 
discussion  about  "distinguishing  thinking  from  knowing,"  she  establishes  that 

we  are  not  only  capable  of  knowing  - achieving  specific  answers  to  questions  that 
can  then  be  taught  and  learned  in  ways  that  shape  cultures  and  so  human  worlds  - 
but  of  thinking  about  what  we  know,  and  so  also  thinking  beyond 
knowledge... that  [Plato]  questioning  dissolves  certainties,  beliefs  people  hold  as 
knowledge  (conventional  or  formal),  thereby  reopening  minds,  reawakening 
curiosity,  and,  not  incidentally,  drawing  people  back  into  discussion  with  each 
other  that  equalizes  them  as  lovers  of  wisdom  none  of  them  possesses,  (p.  1 13) 

Considering  partial  knowledge  is  important.  Our  courses  reflect  and  perpetuate  a sort  of 
partial  knowledge  from  Minnich's  perspective.  Our  courses  only  access  parts  of  a total 
subject.  Often  that  subject  matter  is  already  predefined  too  narrowly  and  is  presented  in 
such  a way  as  to  imply  that  it  is  the  'whole'  subject  and  uses  definitions  where  anything 
else  is  judged  to  be  wrong  or  inferior,  etc.  Although  I clearly  and  often  state  to  my 
students  that  there  is  so  much  content  for  us  to  choose  from  and  that  we  can  only  engage 
in  making  meaning  about  a selected  set  of  concepts,  topics,  etc.,  the  concept  of  "partial 
knowledge"  brought  about  a deeper  meaning  and  understanding  of  what  I was  trying  to 
express. 

All  of  our  subject  matter  is  complex;  we  must  make  choices  about  what  to  study  with 
students  and  how  deeply  to  consider  the  knowledge  and  its  uses.  That  is  often  difficult, 
and  often  disastrously,  we  rely  on  textbooks  to  make  those  choices  for  us  without 
understanding  or  realizing  the  ramifications.  We  must  realize  that  curricula  can  and 
should  continuously  be  reviewed  and  changed.  This  is  an  exciting  challenge  to  those 
scholarly  and  professional  teaching  professors,  the  inquiry-oriented  scholar  who  practices 
the  art  of  translation,  but  is  not  always  easy  for  those  professors  whose  success  is 
dependent  upon  their  established  identity  as  a specialized  authority  on  a discipline. 
However,  when  considering  that  those  who  teach,  especially  scholars,  should  be  used  to 
conceptual  diversities  in  their  fields,  there  should  be  little  difficulty  when  confronted  with 
change,  regardless  of  whether  change  is  with  course  content,  teaching  methods,  and  new 
inquiry.  Professionals  are  or  should  be  oriented  and  excited  by  the  challenges  it  presents 
and  its  potential  results.  However,  sometimes  professional  identities  (or  lack  of),  attitudes 
and  ambitions,  personal  tastes  and  styles  do  get  in  the  way  of  being  a professional. 
Sometimes  the  practice  of  tolerating  differences  among  ourselves  and  our  students  gets  in 
the  way  of  engaging  in  authentic  communication  to  transform  knowledge,  whether  as 
scholars  or  as  leaders  of  student  learning.  Thinking  is  entwined  with  acting. 


81 


Finally,  in  considering  the  transformation  of  knowledge,  one  must  realize  that  it  is 
relative.  Making  meaning  depends  upon  how. 

Teaching  rests  on  the  faith  that,  although  we  differ  and  may  have  different 
stakes  in  learning  and  in  what  is  available  to  be  learned,  we  can  nevertheless 
approach  comprehension  even  of  that  which  appears  to  be  utterly  outside  our 
own  experience.  How  else  can  we  translate  the  past  into  terms  accessible  to 
newcomers,  to  students?  How  else  can  we  study  differing  schools  of  thought, 
let  alone  cultures?  Translation  across  such  lines  is  possible,  and  the  effort  to 
achieve  it  rewards  us  not  by  giving  us  more  of  ‘the  same,’  but  by  revealing 
mutually  enriching  differences  along  with  discernible  similarities.  Good 
teachers  and  good  scholars  are  not  restricted  to  teaching  their  own  perspectives, 
views,  espoused  theories,  or  cultures.  They  are,  pre-eminently,  creative 
translators.  It  is  a way  of  thinking,  the  way  which  imposes  itself  when  the  object 
is  'different,'  and  requires  us  to  transform  ourselves.  [When  we]  let  ourselves  be 
taught  by  another  culture. . .a  new  organ  of  understanding  is  at  our  disposal— we 
have  regained  possession  of  that  untamed  region  of  ourselves,  unincorporated  in 
our  own  culture,  through  which  we  communicate  with  other  cultures.  (Merleau- 
Ponty,  1964,  p.  120) 

Important  to  consider  is  Minnich's  (2005)  presentation  about  faculty  generalizations  and 
hierarchic  monism,  meaning  that  some  (most)  of  us  have  a sex/gender,  class,  and  race 
implicating  stratification  system  - kinds  of  human  beings  and  that  this  affects  our 
relations.  In  my  opinion,  this  realization  can  free  us  or  it  can  bind  us  in  our  curricula  and 
expectations  of  each  other  and  students,  and  it  can  be  reflected  in  our  teaching. 

Transformative  Learning 

Learning  as  transformation  is  also  an  important  viewpoint  to  consider.  Although  still 
focused  on  "making  meaning,"  it  is  more  from  the  perspective  of  making  meaning  as  a 
learning  process.  Knowledge  is  not  permanently  definitive,  as  there  are  no  fixed  truths. 
Making  meaning  occurs  and  depends  upon  our  level  of  awareness  and  understanding,  the 
conditions  that  exist,  our  level  of  development,  child  to  adult.  Bruner  (1996)  identified 
four  modes  of  making  meaning: 

1.  establishing,  shaping,  and  maintaining  intersubjectivity; 

2.  relating  events,  utterances,  and  behavior  to  the  action  taken; 

3.  construing  of  particulars  in  a normative  context  - deals  with  meaning  relative 
to  obligations,  standards,  conformities,  and  deviations; 

4.  making  propositions— application  of  rules  of  the  symbolic,  syntactic,  and 
conceptual  systems  used  to  achieve  decontexualized  meanings,  including  rules 
of  inference  and  logic  and  such  distinctions  as  whole-part,  object-attribute, 
and  identify-othemess.  (p.  93) 

A fifth  mode  of  making  meaning  was  added  to  Bruner's  list  by  Transformation  Theory. 

5.  becoming  critically  aware  of  one's  own  tacit  assumptions  and  expectations 
and  those  of  others  and  assessing  their  relevance  for  making  an  interpretation. 
[Mental  models,  existing  frameworks,  schema,  etc.]. 
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In  our  initiative,  we  have  chosen  Bloom's  Revised  Taxonomy  of  Cognitive  Learning 
(Anderson  & Krathwohl,  2001).  Kitchener  (1983)  suggests  three  levels  of  cognitive 
processing: 

At  the  first  level,  individuals  compute,  memorize,  read  and  comprehend.  At  the 
second  level  [metacognition],  they  monitor  their  own  progress  and  products,  as 
they  are  engaged  in  first-order  cognitive  tasks. . .The  third  level. . . [the]  epistemic 
cognition,  must  be  introduced  to  explain  how  humans  monitor  their  problem 
solving  when  engaged  in  ill- structured  problems,  i.e.  those  which  do  not  have  an 
absolutely  correct  solution.  Epistemic  cognition  has  to  do  with  reflection  on  the 
limits  of  knowledge,  the  certainty  of  knowledge,  and  the  criteria  for  knowing. . . . 
Epistemic  cognition  emerges  in  late  adolescence,  although  its  form  may  change 
in  the  adult  years,  (p.  230) 

We  learn  as  we  go  through  the  process  of  using  a prior  interpretation  to  construe  a new  or 
revised  interpretation  of  the  meaning;  this  can  guide  future  action.  Rosenfeld  (1988) 
describes  this  as  the  appropriation  of  symbolic  models  that  are  composed  of  images  and 
conditioned  affective  reactions  from  our  culture  or  upbringing,  our  individual  frames  of 
reference.  We  then  make  analogies  to  interpret  the  new  meanings  of  new  experiences. 
Eearning  can  be  intentional  and  can  be  the  result  of  deliberate  inquiry;  it  can  also  be 
incidental  or  serendipitous.  One  can  be  fully  aware  when  learning  takes  place  or  be 
completely  unaware.  In  learning,  we  use  language  to  articulate  it  to  ourselves  or  others. 
However,  there  is  a type  of  learning  where  words  are  not  necessary  to  make  meaning, 
that  of  presentational  learning  (e.g.,  experiencing  motion,  color,  aesthetic,  empathy, 
appreciation).  Eanguage  is  used  when  we  want  to  share  an  experience.  Beliefs  are  not 
usually  encoded  with  words  but  rather  by  interactions  and  generalizations. 

Weiss  (1997)  says: 

Indeed,  research  into  the  unconscious  acquisition  of  knowledge  demonstrates 
that  the  human  being  has  an  enormous  capacity  nonconsciously  to  make 
inferences  from  complex  data,  to  solve  difficult  puzzles,  and  to  make  broad 
generalizations  from  particular  experiences. . . 

[T]he  nonconscious  capacity  of  people  to  acquire  information  is  much  more 
sophisticated  and  rapid  than  their  conscious  capacity  to  do  this.  Also  human 
beings  have  no  conscious  access  to  the  nonconscious  process  that  they  use  to 
acquire  information.  People  cannot  describe  them;  they  are  conscious  only  of  the 
results  of  their  nonconscious  mental  activities,  (p.  428) 

There  are  a lot  of  ways  to  make  meaning.  Cognition  has  strong  affective  and  cognitive 
dimensions  in  which  a person's  sensitivity  and  responsiveness  interact  to  invent,  discover, 
interpret,  and  transform  meaning.  Transformative  learning  can  be  a highly  emotional  and 
intense  experience.  This  is  often  when  we  become  aware  of  the  need  to  change.  Important 
to  note  is  that  the  "who,  what,  when,  where,  why,  and  how  of  learning  may  be  understood 
as  situated  in  a specific  cultural  context"  (Mezirow  as  cited  in  Mezirow  et  ah,  2000,  p.  7). 
We  are  connected  as  humans;  that  is  common  with  the  desire  to  understand.  However, 
culture  either  enables  or  inhibits  the  realization  of  common  interests,  methods  of 
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communicating,  and  the  realization  of  learning  capabilities.  Langer  (1997)  defines 
learning  mindfully  as  "the  continuous  creation  of  new  categories,  openness  to  new 
information,  and  an  implicit  awareness  of  more  than  one  perspective,"  whereas 
mindlessness  learning  "involves  relying  on  past  forms  of  action  or  previously  established 
distinctions  and  categories"  (p.  4).  This  resonates  with  me,  especially  in  light  of  what  we 
are  trying  to  accomplish  through  this  initiative.  Transformative  learning  is  a process 
where  we  change  or  transform  our  pre-existing  frames  of  reference  or  schemata  (e.g. 
mindsets,  assumptions);  where  we  extend  them  to  be  more  inclusive,  open  to  change,  and 
reflective;  and  where  we  begin  to  generate  different  beliefs  or  opinions  more  true  present 
than  our  past  ones.  Through  discourse,  we  construct  new  meanings  from  the  experiences 
of  others  and  then  take  actions  based  upon  new  insights.  Transformative  Theory  moves 
us  to  negotiate  and  take  action  on  our  own  purposes,  values,  and  meanings  rather  than 
merely  acting  on  those  of  others;  therefore,  we  gain  control  over  our  own  lives  and  begin 
to  make  our  own  decisions. 

Mezirow  discusses  Habermas's  (1984)  domains  of  learning,  instrumental  and 
communicative.  Instrumental  learning  relates  to  learning  to  control  or  manipulate  the 
environment  or  other  people,  where  there  could  be  task-oriented  problem  solving  to 
improve  performance.  Communicative  learning  is  more  focused  on  determining  what 
others  mean  when  they  communicate  with  you,  establishing  their  values,  intentions,  or 
moral  issues.  This  engages  individuals  in  the  assessment  of  meanings  behind  another's 
words  to  determine  assumptions,  truths,  or  appropriateness.  It  might  involve  establishing 
the  qualifications  of  the  speaker  and  their  authenticity.  For  example,  when  others  are 
directing  us  at  work,  we  need  to  determine  if  they  are  authorized  to  do  so.  When 
determining  the  assumptions  underlying  someone's  words,  one  has  to  consider  intent, 
wisdom,  worldview,  literal  or  metaphoric  meaning,  and  more.  Most  of  our  learning 
involves  both  domains.  Reflective  discourse,  from  Transformational  Theory,  is  the  use  of 
dialogue  to  determine  common  understanding  or  the  assessment  of  a justification  of  an 
interpretation.  It  involves  critical  assessment,  especially  of  assumptions,  and  often  results 
in  clearer  understanding.  Emotional  maturity  is  required  if  discourse  is  to  be  effective. 
Goleman  (1998)  and  others  have  constructed  what  is  known  as  emotional  intelligence  or 
EQ  as  when  one  can  manage  emotions  well  as  well  as  handle  relationships  effectively 
and  can  recognize  and  motivate  positive  results  from  others’  emotions:  clear  thinking, 
empathy,  self-control  and  trustworthiness.  Goleman  claims  that  most  of  the  success  at 
work  is  a direct  result  of  EQ.  The  culture  of  the  U.S.  is  argumentative,  where  we  compete 
individually  to  win  or  lose.  It  is  not  the  best  culture  for  collaborative  thinking,  and 
although  one  of  our  strengths  as  a nation,  conversely,  it  is  also  one  of  our  weaknesses. 
Effective  discourse  is  not  based  upon  winning  arguments  but  rather  on  the  "trying  on"  of 
other  viewpoints.  Bruner's  (1990)  definition  of  open  mindedness  is  "a  willingness  to 
construe  knowledge  and  values  from  multiple  perspectives  without  loss  of  commitment 
to  one's  own  values"  (p.  30).  It  requires  "epoch"  or  "a  provisional  suspension  of  judgment 
about  the  truth  or  falsity  of,  or  the  belief  or  disbelief  in,  ideas  until  a better  determination 
can  be  made"  (p.  31).  To  most  effectively  engage  in  discourse  where  there  is  open 
mindedness,  participants  must  have  the  following  ideal  conditions: 

• more  accurate  and  complete  information 

• freedom  from  coercion  and  distorting  self-deception 
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• openness  to  alternative  points  of  view:  empathy  and  concern  about  how 
others  think  and  feel 

• the  ability  to  weigh  evidence  and  assess  arguments  objectively 

• greater  awareness  of  the  context  of  ideas  and,  more  critically, 
reflectiveness  of  assumptions,  including  their  own 

• an  equal  opportunity  to  participate  in  the  various  roles  of  discourse 

• willingness  to  seek  understanding  and  agreement  and  to  accept  a 
resulting  best  judgment  as  a test  of  validity  until  new  perspectives, 
evidence,  or  arguments  are  encountered  and  validated  through  discourse 
yielding  a better  judgment.  (Mezirow,  2005,  p.l3) 

Meaning  structures,  or  frames  of  reference  (schemas),  are  those  (models)  structures  of 
assumptions  and  expectations  used  to  filter  our  impressions.  These  shape  or  delimit  our 
cognition  and  disposition  and  provide  our  context  for  making  meaning.  Frames  of 
reference  change  as  we  construe  meanings.  A frame  of  reference  has  two  elements,  a 
habit  of  mind  - the  set  of  assumptions  or  predispositions  acting  as  our  filters  - and 
resulting  points  of  view.  The  following  are  examples  of  habits  of  mind: 

• Sociolinguistic  (cultural,  ideological,  social  norms,  customs,  language  games, 
etc.) 

• Moral-ethical  (conscience,  moral  norms) 

• Epistemic  (learning  styles,  sensory  preferences,  focus  on  whole  or  parts;  concrete 
or  abstract) 

• Philosophical  (religious,  philosophical,  transcendental  world  view) 

• Psychological  (self-concept,  personality  types,  repressed  parental  prohibitions 
dictating  feelings  and  attitudes,  emotions,  etc.) 

• Aesthetic  (values,  tastes,  attitudes,  standards,  and  judgments  about  beauty;  insight 
and  authenticity  of  aesthetic  expressions,  the  humorous,  etc.)  (Mezirow,  2005,  p. 
16) 

These  can  generally  be  described  as  conservative  or  liberal  in  orientation:  the  tendency  to 
move  toward  people  or  away;  the  way  one  approaches  the  unknown,  fearfully  or  with 
confidence;  the  preference  for  being  alone  or  with  others;  ethnocentricity  - viewing  others 
who  are  different  negatively  or  interestingly;  tendency  to  challenge  authority  or  respect  it, 
focusing  on  a problem  holistically  or  in  parts;  and  so  much  more.  A habit  of  mind  is 
expressed  as  a point  of  view  where  clusters  of  meaning  schemes  are  comprised  of 
expectations,  beliefs,  feelings,  attitudes,  judgments,  etc.  These  are  usually  outside  of 
awareness  and  occur  subconsciously  unless  engaged  in  critical  reflection.  We  are 
"anchored"  in  our  frames  of  references.  They  establish  our  identity,  and  we  judge  other 
points  of  view  against  them.  Learning  is  often  perceived  as  the  effort  to  add  to  our  "fixed" 
frames  of  reference.  Transformative  learning  changes  this  learning  predisposition  and 
enables  one  to  become  more  flexible,  more  inclusive,  less  narrow,  emotionally  capable  of 
change,  and  through  doing  so,  much  more  dependable.  Transformations  can  occur  in  four 
ways: 

1 . by  elaborating  existing  frames  of  reference 

2.  by  learning  new  frames  of  reference 

3.  by  transforming  points  of  view 
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4.  by  transforming  habits  of  mind  (Mezirow,  2005,  p.  18) 

Transformation  occurs  over  time,  during  which  individuals  reformulate  structures  of 
meaning.  This  happens  as  they  reconstruct  dominant  narratives  or  predispositions. 
Possibly,  the  process  of  reconstruction,  the  dispositional  orientation  or  openness  to 
change,  could  be  considered  a fifth  way  that  transformations  occur.  The  capability  of 
critical  reflection  positions  one  to  transformational  learning.  There  are  three  assumptions 
essential  for  critical  reflections: 

• paradigmatic  assumptions  that  structure  the  world  into  fundamental  categories 
(the  most  difficult  to  identify  in  oneself) 

• prescriptive  assumptions  about  what  we  think  ought  to  be  happening  in  a specific 
situation 

• causal  assumptions  about  how  the  world  works  and  how  it  may  be  changed  (the 
easiest  to  identify)  (Mezirow,  2005  , p.  19) 

When  transformative  learning  occurs,  we  are  able  to  make  a frame  of  reference  more 
dependable  because  it  enables  us  to  better  justify  our  opinions  and  interpretations  when 
making  meanings.  Since  frames  of  references  are  highly  individual,  transformative 
learning  can  be  perceived  as  a way  of  problem  solving  where  one  engages  in  reframing 
the  problem  or  redefining  it  and  then  engaging  in  critical  reflection  of  the  underlying 
assumptions.  Any  new  perspective,  however,  does  need  to  be  justified  through  discourse 
with  others.  Critical  reflection  may  engage  us  in  questioning  the  content,  process, 
premises  or  underlying  assumptions,  etc.  For  example,  in  our  context,  we  could  question 
whether  we  have  assigned  reasonable  performance  indicators  for  particular  academic 
tasks  to  be  performed  by  students.  Are  they  fair,  representative?  Critical  reflection 
involves  reasoning  and  intuition  rather  than  assimilating  mindlessly.  Cohen  (1997) 
described  a student  transformation  in  which  a student  had  negative  learning  experiences 
but,  through  a series  of  transformational  learning  events,  became  more  secure  as  a 
learner,  moving  from  that  insecurity  to  "I  can  understand. . . " and  "I  am  a smart, 
competent  person. . . " - a habit  of  mind.  When  we  try  on  another's  point  of  view,  we 
change  our  own. 

Kegan  offers  a constructivist  approach  to  transformative  learning: 

1 . Transformational  kinds  of  learning  need  to  be  more  clearly  distinguished  from 
informational  kinds  of  learning,  and  each  needs  to  be  recognized  as  valuable  in 
any  learning  activity,  discipline,  or  field. 

2.  The  form  that  is  undergoing  transformation  needs  to  be  better  understood;  if  there 
is  no  form,  there  is  no  transformation. 

3.  At  the  heart  of  a form  is  a way  of  knowing  (frame  of  reference);  thus  genuinely 
transformational  learning  is  always  to  some  extent  an  epistemological  change 
rather  than  merely  a change  in  behavioral  repertoire  or  an  increase  in  the  quantity 
or  fund  of  knowledge. 

4.  Even  as  the  concept  of  transformational  learning  needs  to  be  narrowed  by 
focusing  more  explicitly  on  the  epistemological,  it  needs  to  be  broadened  to 
include  the  whole  life  span;  transformational  learning  is  not  the  province  of 
adulthood  or  adult  education  alone. 
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5.  Adult  educators  with  an  interest  in  transformational  learning  may  need  a better 
understanding  of  their  students'  current  epistemologies  so  as  not  to  create  learning 
designs  that  unwittingly  presuppose  the  very  capacities  in  the  students  their 
designs  might  seek  to  promote. 

6.  Adult  educators  may  better  discern  the  nature  of  learners'  particular  needs  for 
transformational  learning  by  better  understanding  not  only  their  students'  present 
epistemologies  but  the  epistemological  complexity  of  the  present  learning 
challenges  they  face  in  their  lives,  (as  cited  in  Mezirow,  2005,  p.  47) 

Figure  A. 6. 13:  Two  Kinds  of  Learning:  Informative  and  Transformative 

A\  Informative:  Changes  in  whatv/6  know 


B) 


Transformative:  Changes  in  how\N&  know 


(Mezirow  et  ah,  2000,  p.  50) 

Epistemology  refers  to  "not  what  we  know  but  our  way  of  knowing”: 

• “meaning  forming"  - the  activity  by  which  we  shape  a coherent  meaning  out  of 
the  raw  material  of  our  outer  and  inner  experiencing.  Constructivism  recognizes 
that  reality  does  not  happen  preformed  and  waiting  for  us  merely  to  copy. . .Our 
perceiving  is  simultaneously  an  act  of  conceiving,  of  interpreting.  ..Our 
experience  is  less  what  happens  to  us  and  more  what  we  make  of  what  happens  to 
us. 

• “reforming  our  meaning  -forming”  - a metaprocess  that  affects  the  very  terms  of 
our  meaning-constructing.  We  do  not  only  form  meaning,  and  we  do  not  only 
change  our  meanings;  we  change  the  very  form  by  which  we  are  making  our 
meanings.  We  change  our  epistemologies.  (Kegan,  as  cited  in  Mezirow,  2005,  p. 
52) 
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Therefore,  “transformational  learning  supports  changes  in  a learner's  form  of 
knowing,. . .the  psychological  process  of  transformations  in  our  knowing”  (p.  53).  Other 
authors  provide  a deeper  look  at  transformative  learning  from  various  critical  and 
analytical  perspectives  - well  worth  the  reading.  Let  us  end  with  the  distinction  between 
transformative  learning  and  critical  reflection.  Brookfield  (1987)  defines  the  relationship 
between  the  two:  critical  reflection  is  integral  to  transformative  learning.  Transformative 
learning  cannot  happen  without  critical  reflection  at  every  stage,  but  they  are  not 
synonyms  for  each  other.  Just  because  critical  reflection  occurs,  does  not  mean  that 
transformative  learning  ensues.  If  questioned  assumptions  remain  the  same  after  critical 
reflection,  then  perhaps  transformative  learning  has  not  been  achieved.  And  beyond  that, 
there  should  be  a result  of  transformative  actions.  "Deciding"  can  be  an  action  in 
transformative  learning  (as  cited  in  Mezirow,  2005,  p.  141).  Taylor  presents  a study  of 
transformation  and  identifies  the  dimensions  from  the  learner's  perspective.  The  five 
dimensions  are  1)  Toward  Knowing  as  a Dialogical  Process;  2)  Toward  a Dialogical 
Relationship  with  Oneself;  3)  Toward  Being  a Continuous  Learner;  4)  Toward  Self 
Agency  and  Self  Authorship;  and  5)  Toward  Connection  with  Others.  Brookfield  further 
defines  each  one.  These  enlighten  us  as  to  how  to  teach  with  developmental  intentions. 
This  is  adult  learning  focused.  Cranton  (as  cited  in  Mezirow)  discusses  individual 
differences,  transformative  learning  and  the  educator's  roles:  1)  "a  responsibility  to  assist 
learners  in  becoming  aware  of  their  psychological  preferences";  2) "encouraging  critical 
questioning  of  psychological  habits  of  mind  and  supporting  the  differentiation  of  the 
individual  from  the  collective";  and  3)  “educators  need  to  help  create  learning 
experiences  that  involve  learners  of  different  predispositions  in  that  process"  (p.  195). 
Yes!  This  is  what  professional  teachers  or  leaders  of  learning  are  to  strive  to  do. 

Critical  Thinking  (CT)  and  Critical  Reflection  (CR) 

Many  authors,  (especially  in  the  1980s)  have  considered  critical  thinking  and  how  to 
develop  students'  ability  to  think  more  critically  so  learning  is  achieved  at  higher 
cognitive  levels  and  understanding  deepens  and  broadens  (Bow ell  & Kemp,  2005; 
Browne  and  Keeley,  1986;  Dauer,  1989;  Meyers,  1986;  Nosich,  2005;  Vaughn,  2005  and 
others).  Definitions  by  leading  researchers  include 

• Critical  thinking  is  reasonable,  reflective  thinking  that  is  focused  on  deciding 
what  to  believe  or  do.  (Ennis,  1987) 

• Critical  thinking  is  skillful,  responsible  thinking  that  is  conducive  to  good 
judgment  because  it  is  sensitive  to  context,  relies  on  criteria,  and  is  self- 
correcting.  (Lipman,  1995) 

• Critical  thinking  is  thinking  about  your  thinking,  while  you're  thinking,  in  order  to 
make  your  thinking  better.  (Paul,  1992) 

So,  critical  thinking  is  reflective  and  involves  standards  and  being  responsible.  Three 
parts  of  CR  are  that  it  involves  asking  questions,  reasoning  out  the  answers,  and 
ultimately  believing  the  results  of  our  reasoning.  One  must  develop  the  abilities  to  ask  the 
right  questions  and  reason,  but  in  considering  the  belief  aspect,  there  are  four  indicators 
of  when  we  do  not  believe  the  results  of  our  reasoning: 


88 


• I reason  something  out,  but  strong  emotions  arise  within  me  against  the  result. 

• I find  myself  believing  contradictory  things. 

• I believe  something  very  strongly,  but  I find  I am  unable  to  come  up  with  good 
reasons  for  the  belief.  In  fact,  I don’t'  think  I even  need  reasons.  Thinking  the 
opposite  seems  ridiculous. 

• I reason  something  out,  but  my  actions  do  not  follow  my  reasoning.  (Nosich, 
2005,  p.  11) 

Critical  thinking  is  not: 

1.  negative  - although  often  perceived  "being  critical"  means  being  negative.  In  CR, 
it  has  no  negative  meaning  at  all.  It  is  related  to  "criteria. . .thinking  that  meets 
high  criteria  of  reasonableness. . .to  learn  to  think  things  through,  and  to  think 
them  through  well:  accurately,  clearly,  sufficiently,  reasonably... effective 
thinking. . . [it]  does  not  involve  making  judgments. . . [T]o  be  judgmental  is 
certainly  not  to  be  a critical  thinker." 

2.  emotionless  thinking  - some  emotions  can  impair  critical  thinking,  e.g.  rage, 
panic;  others,  however,  help,  e.g.  love  of  truth,  joy  of  discover,  anger  at  biased 
interpretations,  fear  of  making  an  unreasonable  decision  when  something  is 
important.  Emotions  are  essential  in  that  they  can  give  us  data. . . [It]  often  would 
be  foolhardy  to  ignore  the  data  from  emotions;  also,  being  logical  is  linked  to 
having  feeling.  (Nosich,  2005,  p.  13) 

Figure  A. 6. 14:  The  Circle  of  Elements 
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(Nosich,  2005,  p.  13  ) 
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Figure  A. 6. 15:  Critical  Thinking  Process 


(Nosich,  2005,  p.  14) 

Critical  thinking  goes  beyond  problem  solving,  but  involves  it.  Asking  questions  is 
fundamental  to  CR.  One  needs  to  know  when  a question  should  be  asked  or  be  able  to 
identify  when  a problem  needs  to  be  solved,  which  takes  skill.  CR  begins  with  posing  the 
problem.  Impediments  to  CR  are 

1 . Forming  a picture  of  the  world  on  the  basis  of  news 

2.  Forming  a picture  of  the  world  on  the  basis  of  movies,  TV,  advertising, 
magazines 

3.  All-or-nothing  thinking  (black-or- white  thinking);  Us-versus-them  thinking. 
Stereotyping 

4.  Fears 

5.  Some  educational  practices  discourage  CR 

a.  Student  role  is  passive  recipient  of  knowledge 

b.  Student  role  is  to  memorize  and  regurgitate  information 

c.  Teacher's  role  is  to  dispense  information 

d.  Questions  on  exams  should  be  taken  only  from  what  is  covered  in  class 

e.  Problems  assigned  to  students  should  always  be  clearly  formulated 

f.  There  is  an  adequate  answer  to  every  question 

g.  Everything  is  just  a matter  of  opinion 

6.  Egocentrism  stands  in  the  way  of  empathy;  causes  us  to  make  judgments 
based  upon  self-interests;  makes  it  difficult  to  determine  accuracy  from 
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inaccuracy;  makes  one  misunderstand  other  people's  motives  as  well  as  our 
own.  For  teachers,  it  can  lead  to  seeing  education  in  terms  of  grades  only, 
missing  other  benefits  of  education 

7.  Developmental  patterns  of  thinking  - assumptions  one  makes  and  lives  by,  e.g. 
previous  commitments,  previous  personal  experience-the  ability  to  think  in  an 
unbiased  way,  using  evidence,  rather  than  basing  views  upon  past  experiences 
and  interpreting  only  in  that  light  rather  than  the  new  evidence  or  context.  Not 
being  ruled  by  predispositions.  (Nosich,  2005,  p.  23)  (source) 

Nosich  (2005)  leads  one  through  an  understanding  of  the  need  for  CR,  its  versatility,  and 
its  value  to  all  levels  of  thinking.  He  offers  a model  that  provides  the  elements  of 
reasoning,  the  standards  of  reasoning,  and  considers  the  fields  where  CR  takes  place.  It  is 
a great  resource  for  our  endeavor. 

Bowell  and  Kemp  (2005)  explore  CR,  addressing  why  we  need  to  become  critical 
thinkers  and  implications  for  our  students.  They  discuss  the  role  of  logic  as  deductive 
validity  and  inductively  as  a force  and  thoroughly  address  the  construction/reconstruction 
of  argument,  wrapping  with  "truth,  knowledge,  and  belief,"  tying  into  Nosich  (2005) 
above.  Their  address  is  more  technical  but  serves  as  a good  resource  in  that  vein. 

To  continue  from  the  perspectives  of  others  - critical  thinking  is  how  someone  thinks, 
what  causes  a belief,  and  how  one  engages  in  determining  the  value  of  a belief.  It  is  “the 
systematic  evaluation  or  formulation  of  beliefs,  or  statements,  by  rational  standards"  and 
involves  logic,  "the  study  of  good  reasoning,  or  inference,  and  the  rules  that  govern  it" 
(Bowell  & Kemp,  2005,  p.  7).  However,  critical  thinking  involves  more  than  logic;  it  also 
involves  establishing  the  truth  or  falsity  of  beliefs,  the  evaluation  of  arguments,  evolving 
evidence  through  analysis  and  investigation.  Critical  thinking  skills  can  lead  one  to  new 
knowledge,  deepen  understanding,  and  more  importantly  empower  an  individual.  It  is  a 
prerequisite  for  enabling  students  to  engage  in  problem  solving,  active  learning,  and 
intelligent  self-improvement  or  intellectual  growth.  It  is  important  because  it  guides  our 
actions.  It  is  "thinking  outside  the  box."  Critical  thinking  requires  skills  in  argument, 
reasoning,  and  inferencing.  Arguments  are  a statement  or  statements  supposedly 
providing  reasons  for  accepting  another  statement. . .the  main  focus  of  critical  thinking. 
Those  reasons  are  known  as  "premises."  Therefore,  arguments  are  "a  group  of  statements 
in  which  some  of  them  (the  premises)  are  intended  to  support  another  of  them  (the 
conclusion)”  (Vaugh,  2005,  p.  4). 

Meyers  (1986)  departs  from  traditional  perspectives  and  posits  that  critical  thinking  is 
different  across  disciplines.  While  acknowledging  that  most  definitions  are  usually  based 
upon  formal  or  informal  logic  or  problem  solving  skills,  these  perspectives,  he  feels,  have 
limitations  that  do  not  transfer  to  other  contexts:  for  example,  the  rules  of  formal  logic 
will  not  apply  to  an  analysis  of  a Picasso  painting  or  novel.  Also  the  logic  and  problem 
solving  approaches  imply  that  there  is  always  a problem  and  solution.  Meyers  feels  the 
theory  suggests  that  once  logic  and  problem  solving  skills  are  mastered,  then  students  can 
easily  apply  them  across  disciplines,  but  there  is  no  assurance  of  that  across  disciplines. 
Finally,  he  feels  that  one  of  the  serious  difficulties  with  the  logic  and  problem  solving 
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approach  to  critical  thinking  is  that  of  "passing  the  buck";  in  other  words,  if  students  can 
be  "channeled  into  courses  in  logic  or  problem  solving,  other  teachers  can  relax  because 
the  difficult  job  of  teaching  students  how  to  think  will  be  done  elsewhere"  (p.  9).  Instead 
he  believes  all  teachers  have  the  responsibility  and  play  an  essential  role  in  the 
development  of  students'  critical  thinking,  each  from  his/her  own  discipline's  perspective. 
Students  need  to  be  challenged  to  practice  critical  and  analytical  thinking  across  contexts 
and  all  disciplines  studied  (p.  9). 

In  synchrony  with  the  discussion  above  on  critical  reflection,  Meyers  (1986)  mentions 
Dewey's  ([1910],  1982)  definition  or  essence  of  critical  thinking  as  "suspended 
judgment,"  or  healthy  skepticism.  ..  the  qualities  of  'reflective  thought'  that  might  also  be 
said  to  characterize  critical  thinking. . .Active,  persistent,  and  careful  consideration  of  any 
belief  or  supposed  form  of  knowledge  in  light  of  the  grounds  that  support  it. . .constitutes 
reflective  thought"  ([p.7]  p.8).  Loacker  et  al.  (1984)  went  on  to  describe  it  as  an 
experience  in  which  students  "questions,  examine,  prod,  poke,  dissect,  and  explicate" 
(p.3).  This  process  requires  that  students  struggle  with  problems  and  issues  and, 
important  to  note,  that  they  see  their  professors  doing  the  same.  Also  interesting  to  note  is 
that  the  authors  feel  that  if  students  engage  in  the  rediscovery  of  what  is  already  known, 
they  will  have  little  motivation  for  critical  analysis.  They  suggest  that  college  professors 
create  classrooms  where  there  is  an  atmosphere  that  encourages  natural  inquisitiveness. 
"Humans  are  bom  smart. . .all  we  have  to  do  is  to  stop  doing  things  that  make  them 
stupid"  (Holt,  1982,  p.  161).  This  means  interactive  classrooms,  where  the  students  are 
not  sponges  for  lectures  and  where  students  can  engage  in  the  subject  matter  and  practice 
the  art  of  critical  thinking.  An  analogy  presented  was  the  idea  of  a college  course  in 
basketball  where  students  learn  terminology,  diagram  plays,  and  watch  videos;  then  for 
the  final  exam,  they  are  expected  to  play  a competent  game.  They  make  the  point  that  this 
is  what  is  expected  when  professors  present  lectures  on  theory,  test  for  recall,  and  then 
expect  students  to  demonstrate  good  critical  thinking  skills  on  the  final  project.  It  does 
not  “play”  out  that  way. 

In  traditional  teaching  there  is  often  an  implicit  assumption  that  learning  to  think 
critically  develops  naturally  as  students  learn  increasingly  complex  levels  of  a 
discipline  content  and  information.  While  there  may  be  a natural  basis  for  human 
inquisitiveness,  there  is  nothing  natural  about  learning  a framework  for  analyzing 
a modern  novel  or  management  system.  Analytical  frameworks  must  be  taught 
explicitly  and  constructed  consciously,  beginning  with  simple  operations  and 
building  toward  complexity  and  subtlety.  Initially,  for  most  students,  this  means 
learning  to  recognize  key  concepts,  terms,  issues,  and  methodologies  - not  by 
memorizing  them  but  by  working  with  them  in  the  context  of  real  problems  and 
concerns  and  by  relating  them  to  experiences  and  previous  learning.  (Meyers, 
1986,  p.  10) 

Meyers  (1986)  goes  on  to  discuss  student  interest  and  motivation,  including  barriers  to 
student  interest.  He  discusses  how  students  create  meaning  in  their  own  terminology  and 
encourages  the  use  of  analogies  and  metaphors.  He  discusses  structuring  classes  to 
promote  critical  thought  by  balancing  content  and  process,  stating  that  when  using 
textbooks,  instructors  feel  obliged  to  complete  the  entire  text  at  the  expense  of  process. 
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This  has  real  meaning  for  our  initiative.  We  are  firmly  committed  to  changing  the  process 
models  so  that  does  not  happen.  He  confirms  that  it  is  important  to  begin  with  "what  do 
you  want  students  to  know  and  be  able  to  do"  (p.  55),  which  makes  it  possible  to 
determine  what  is  essential  to  cover  in  the  course.  (Our  professors  completed  a thorough 
analysis  process  to  do  exactly  that.)  Meyers  goes  on  to  describe  the  importance  of 
balancing  lecture  and  interaction,  relying  on  Piaget's  learning  theory  on  "social 
transmission  in  the  development  of  new  mental  structures"  (p.56)  and  on  Lawson  and 
Renner  (1975). 

In  order  for  the  learner  to  be  shaken  from  his  egocentric  views,  he  must 
experience  the  viewpoints  and  thoughts  of  others. . .If  he  does  not,  he  has  no 
reason  to  alter  the  mental  structures  that  he  initially  acquired  from  his  self- 
centered  frame  of  reference.  Social  interactions  can  lead  to  conflict,  debate, 
shared  data,  and  the  clear  delineation  and  expression  of  ideas,  (p.  338) 

These  authors,  along  with  Piaget  (1976)  and  Kohlberg,  strongly  believe  that  active  forms 
of  learning  are  more  effective  than  lecture  to  stimulate  cognitive  and  ethical  development 
in  students.  This  does  not  eliminate  lecture  but  puts  it  in  its  proper  place.  In  my  opinion, 
it  repositions  the  burden  of  learning  to  students  - back  where  it  belongs.  Yes,  professors 
are  responsible  for  their  own  learning,  but  the  burden  of  learning  for  students  should  lie 
with  students.  Lectures  are  good  to  stimulate  interest,  explain  a concept  or  theory,  raise 
questions,  present  problems,  provide  information  not  available  in  other  forms,  clarify 
concepts  through  analogies  or  examples,  summarize,  but  not  as  used  by  many  professors. 
It  does  not  allow  for  interaction  by  the  students  and  does  not  lend  itself  to  information 
processing,  especially  transformative  learning  as  described  above. 

Meyers  (1986)  goes  as  far  as  to  present  room  organization  to  better  provide  the 
opportunity  for  critical  thinking.  This  has  always  been  one  of  my  strategies.  I have  never 
used  the  standard  classroom  layout  and  have  often  been  criticized,  and  even  reprimanded, 
for  changing  the  room  organization.  In  fact,  interesting  to  note,  is  that  when  our  new  off- 
campus  centers  were  relatively  recently  built,  many  of  the  rooms  had  tables  bolted  to  the 
floor.  When  I asked  for  round  tables  or  tables  that  could  be  moved  into  circles,  the 
administrative  directors  went  ballistic ! They  only  had  round  tables  in  the  lunchroom,  and 
they  did  not  want  to  move  tables  in  the  other  rooms.  For  those  who  know  me,  they 
already  realize  that  my  persistence  paid  off,  and  now  I am  able  to  command  the  type  of 
arrangement  I need  to  accomplish  interactive  learning.  However,  many  would  not  feel  as 
comfortable  being  that  persistent.  Humorous  to  note  is  that  when  business  and  industry, 
those  who  rent  the  rooms  in  our  centers,  began  to  request  round  tables,  there  was  a shift 
in  thought  that  benefited  me.  The  importance  of  this  is  that  our  institutions  often  are  not 
up  to  date  on  the  better  teaching  space  designs,  thus  making  it  difficult  for  us  to  change, 
even  when  we  are  inclined  to  move  towards  a more  interactive  teaching  and  learning 
environment.  Finally,  Meyers  goes  on  to  discuss  other  ways  to  support  engaging  students 
in  critical  thinking  and  justifies  why  critical  thinking  should  be  part  of  every  course  a 
student  takes. 
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Experiential  Learning 


The  segue  into  problem-based  learning  from  the  topic  of  critical  thinking  might  be 
considered  experiential  learning.  Farquharson  (1995)  describes  Kolb's  (1984)  model  as  a 
cycle  of  experiencing/doing,  sharing  reactions  and  observations,  detecting  patterns  and 
inferring  principles,  and  finally,  planning  new  behavior  based  on  principles. 

Figure  A. 6. 16  : The  Cycle  of  Experimental  Learning 


(Farquharson,  1995,  p.  107) 

Clearly  this  model  presents  the  opportunity  to  visualize  the  way  in  which  critical  thinking 
can  be  used  while  engaged  in  active  learning.  The  model  supports  active  learning  and 
learning  by  doing.  In  planning  experiential  learning,  it  is  important  to  give  equal  weight 
or  attention  to  each  aspect  of  the  cycle.  The  experience  is  the  "means  to  the  end,"  not  the 
end  in  itself.  Therefore,  outlining  the  stages  of  the  experience  will  enhance  understanding 
of  what  is  expected.  The  sharing  and  observations  component  provides  the  opportunity 
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for  students  to  make  meaning  and  clarify  their  understanding.  Moving  into  more 
sophisticated  learning,  the  next  stage  of  detecting  patterns  and  inferring  principles 
connects  data  and  principles  more  deeply,  constructing  meanings  when  determining  what 
and  why  something  took  place.  Students  can  then  generate  principles  for  future  action 
based  upon  what  they  have  learned;  thus,  they  enter  into  the  planning  of  new  behaviors  or 
actions  based  upon  the  principles.  They  take  robust  theories  and  plan  new  activities  to 
engage  in  validity  checks  of  principles  derived  from  theories.  This  is  all  highly  oriented 
toward  critical  thinking.  To  accomplish  well  designed  experiential  learning,  students 
must  engage  in  critical  thinking.  Questions  can  be  important  elements  to  facilitate 
inquiry.  Practitioners,  faculty,  and  students  need  to  learn  to  effectively  frame  questions, 
and  to  know  how  to  determine  the  right  questions.  Emancipatory  learning  "is  the 
practitioner's  skill  in  engaging  learning  in  question  posing  and  question  framing" 
(Farquharson,  1995,  p.l62;  see  also  Brookfield,  1987;  Kurfiss,  1988;  Long,  Paradis,  & 
Long,  1981;  Meyers,  1986;  Mezirow,  1990,  1991;  Paul,  1992).  Useful  questions  and 
higher  order  questions  that  promote  synthesis  and  evaluation  are  important.  Farquharson 
proposes  the  following  approach  to  guide  the  process: 

• Cognitive  questions:  Use  to  check  that  learners  understand  basic  information. 
Example:  What  are  the  five  elements  of  the  EDICT  model? 

• Convergent  questions:  Use  to  relate,  compare,  and  combine  information. 
Example:  How  does  the  EDICT  model  compare  with  Kolb's  stages  of  learning 
cycle? 

• Divergent  questions:  Use  to  apply  knowledge  creatively 

Example:  How  could  you  use  the  EDICT  model  to  design  a unit  of  instruction  on 
basic  budgeting? 

• Evaluative  questions:  Use  to  assess  values  and  judgments. 

Example:  What  are  the  strengths  and  limitations  of  using  the  EDICT  approach 
for  planning?  (p.  163) 

Paul  (1992),  director  of  the  Center  for  Critical  Thinking  at  Sonoma  State  University, 
says:  "Critical  thinking  is  that  mode  of  thinking,  about  any  subject,  content  or  problem,  in 
which  the  thinker  improves  that  quality  of  his  or  her  thinking  by  skillfully  taking  charge 
of  the  structures  inherent  in  thinking  and  imposing  intellectual  standards  upon  them"  (p. 
4).  This  puts  the  focus  on  logical  reasoning  abilities.  Habermas  (1979)  thinks  that  critical 
dialogue  has  power  relationships  that  need  to  be  taken  into  account,  and  he  calls  the  way 
statements  are  justified  by  those  in  power  emancipatory  learning  rather  than  critical 
thinking.  Mezirow  (1991)  derives  his  transformative  learning  from  Habermas,  and  he 
states:  "Emancipatory  education  is  about  more  than  becoming  aware  of  one's  awareness. 
Its  goal  is  to  help  learners  move  from  a simple  awareness  of  their  experiencing  to  an 
awareness  of  the  conditions  of  their  experiencing. . .and  beyond  this  to  an  awareness  of 
the  reasons  why  they  experience  as  they  do  and  to  actions  based  on  these  insights"  (p. 
179).  This  requires  the  critical  reflection  we  have  mentioned  so  often.  Farquharson 
(1995)  suggests  that  when  considering  a teaching  intervention  that  one  engages  in  two 
different  types  of  checks:  "1)  ensure  that  the  design  has  taken  into  account  some  of 
Knowle's  (1985)  principles  and  2)  verify  that  the  design  has  included  some  of 
Wlodkowski's  (1991)  ideas  about  ways  to  gain  and  sustain  learners'  attention  and 
motivation  to  learn."  Therefore,  Farquharson  asks  the  following  questions: 
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• Learners'  Self  Concepts 

1 . Is  the  plan  free  of  factors  that  might  make  the  learner  feel  threatened  or 
inadequate? 

2.  Does  the  assessment  of  the  entry-level  knowledge  and  skill  of  the  learners 
seem  reasonably  well  founded? 

• Learners'  prior  experience 

1 . Is  there  a way  to  check  out  what  learners  already  know  about  the  materials  to 
be  learned  so  the  teacher  can  create  ways  to  graft  the  new  learning  onto 
metaphors  drawn  from  learners'  prior  experience? 

2.  Does  the  design  effectively  draw  out  prior  experiences  that  may  contribute  to 
learning  the  new  material? 

3.  Are  there  opportunities  and  structures  that  will  encourage  synergistic  learning 
among  the  members  of  the  learning  group? 

• Learners'  readiness  to  learn 

1 . Is  this  the  moment  when  the  learners  are  ready  to  explore  this  material,  and  if 
not,  what  steps  can  be  taken  to  enhance  their  readiness  to  learn? 

2.  Why  would  these  learners  want  to  learn  this  material  at  this  time? 

3.  Given  the  assessed  motivation  of  the  learners,  is  the  amount  of  material 
appropriate  or  is  there  a danger  of  teaching  them  more  than  they  would  ever 
want  to  know  about  the  topic? 

4.  Would  there  be  any  value  in  delaying  this  teaching  intervention  until  a more 
potent  "teachable  moment"  presents  itself? 

• Life  centered  or  Problem  centered  relevance  and  application  of  the  material 

1 . Is  there  an  element  in  the  design  that  will  let  the  practitioner  know  very  early 
in  the  learning  process  what  particular  applications  individual  learners  have  in 
mind  for  the  material  they  are  learning? 

2.  Are  there  formative  evaluation  systems  in  place  that  will  keep  the  practitioner 
aware  of  the  way  in  which  learners  construct  meaning? 

3.  How  are  learners  applying  what  they  are  learning?  (pp.  166-168) 

Wlodkowski's  (1991)  ideas  about  ways  to  gain  and  sustain  learners'  motivation  to  learn 
are  offered  through  68  practical  strategies  and  the  following  questions  for  considering  the 
teaching/learning  design: 

1 . What  has  been  done  to  foster  a positive  attitude  for  this  experience? 

2.  Does  the  plan  address  the  felt  needs  of  the  learners? 

3.  What  will  stimulate  and  sustain  learner  motivation  during  the  event? 

4.  What  has  been  done  to  maintain  a positive  climate  for  learning? 

5.  Does  the  even  increase  and  sustain  the  learner's  sense  of  competence? 

6.  How  will  the  learning  continue  to  be  reinforced  in  the  future?  (p.  258) 
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When  considering  learners  versus  students,  Farquharson  (1995,  p.  179)  provides  the 
following  role  descriptions: 


Learners 

Students 

Creative 

Obedient 

Powerful 

Parroting 

Curious 

Obsequious 

Interesting 

Poverty-stricken 

Risk  takers 

Overloaded 

Intuitive 

Mind  readers 

Involved 

Our  goal  is  to  engage  students  as  learners  rather  than  as  merely  students. 


Figure  A. 6. 17:  The  Lewinian  Experiential  Learning  Model 


Concrete 

experience 


Testing  implications 
of  concepts  in  new 
situations 


Observations  and 
reflections 


Formation  of 
abstract  concepts 
and  generalisations 


(Kolb,  1984,  p.  21) 
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Figure  A.6.18:  Kolb’s  Model  of  Experiential  Learning 


Concrete 


(Kolb,  1984,  p.  42) 

Figure  A. 6. 19:  Experiential  Learning  as  the  Process  That  Links  Education  Work  and 
Personal  Development 


Personal 

Development 


(Kolb,  1984,  p.  4) 
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Figure  A. 6. 20:  Dewey’s  Model  of  Experiential  Learning 


Purpose 


(Kolb,  1984,  p.  23) 

Problem-based  learning  (PEL)  in  HE  - Learning  With  Complexity 

There  are  a good  number  of  sourees  available  on  problem-based  learning;  however,  I will 
focus  on  one  primary  one,  as  it  does  an  excellent  job  of  justifying,  exploring,  and 
explaining  high  quality  problem-based  learning.  The  academic  beauty  of  problem-based 
learning  should  be  that  there  is  more  than  one  solution  to  any  problem  and  that  students 
are  strongly  encouraged  to  create  and  provide  argument  for  their  particular  solution.  This 
stimulates  and  challenges  them  to  go  beyond  what  they  might  have  considered  if  solution 
justification  was  not  part  of  the  requirement.  However,  students  really  respond  to  the 
challenge  and  enjoy  challenging  professors  and  each  other.  If  the  problems  are  quality 
problems  with  quality  criteria  (rubrics)  to  guide  their  responses,  the  outcomes  are  usually 
high  quality.  Problem-based  learning  also  teaches  students  to  value  their  own 
perspectives  and  voices;  however,  problem-based  learning  can  take  some  adjustment  for 
some  students  who  have  been  immersed  in  lecture-based  courses.  Students  may  initially 
find  it  difficult,  more  complex,  and  that  the  level  of  effort  and  work  on  their  part  is  much 
greater;  thus,  they  may  resist  at  first.  But  once  fully  engaged,  most  students  are 
"converted"  and  eagerly  participate.  Problem-based  learning  demands  a greater 
understanding  of  the  knowledge.  Its  solutions  take  research,  options  must  be  explored, 
and  students  must  develop  an  ability  to  critique  information  and  alternative  solutions. 
Students  must  be  able  to  engage  in  analysis,  justification,  and  reasoning.  If  problems  are 
well  designed,  they  are  complex.  Required  performance  tasks,  across  a wide  range  of 
knowledge  areas  and  skills,  are  nested  within  the  problem.  Students  engage  with 
complexity  and  ambiguities  become  evident.  Most  importantly,  students  are  required  to 
integrate  knowledge  and  skills  from  across  disciplines  and  subjects. 

Professors,  however,  must  reposition  themselves  related  to  teaching.  Using  problem- 
based  learning  is  more  complex  but  equally  more  rewarding  and  changes  the  entire 
teaching-learning  context.  Professors  have  to  re-orient  their  position  with  students.  They 
are  no  longer  the  primary  knowledge  broker.  Textbooks  are  no  longer  the  primary  source 
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and  should  not  have  been  anyway.  Students  become  a primary  source,  and  if 
accomplished  well,  students  challenge  the  perspectives  and  experience  of  the  professor.  I 
have  found  this  to  be  personally  rewarding.  When  students  come  to  me  and  question  my 
knowledge,  directions,  suggestions,  and,  yes  - even  requirements,  if  I cannot  justify  my 
perspective  or  requirement,  then  it  probably  is  not  a good  one!  The  learning  environment 
where  problems  are  the  center  and  source  of  learning  is  without  doubt  the  more  exciting, 
stimulating,  and  rewarding  teaching  environment.  However,  it  is  work,  and  the  professor 
must  prepare  for  it.  Those  professors  who  have  the  interests  of  the  student  at  the  core  of 
their  ethics  and  practice  and  those  that  want  to  continuously  raise  the  learning  bar  for 
themselves  and  their  students  find  that  their  rewards  are  so  integrally  entwined  with  the 
success  of  their  students  that  they  thrive  as  well.  Problem-based  learning  is  a method  that 
works  to  "make  sense"  of  learning  or  "create  meaning"  (Savin-Badin,  2000). 

Savin-Baden  (2000)  feels  that  although  it  (PBL)  has  been  an  approach  to  learning  since 
the  1960-70s,  it  has  yet  to  be  fully  realized  to  achieve  its  full  potential  in  learning 
environments.  The  author  makes  a distinction  between  problem-based  learning  and 
problem  solving  learning.  Problem  solving  learning  has  been  used  for  years.  This  is  when 
professors  give  a lecture  and  then  provide  a set  of  questions.  Students  are  to  find  the 
solutions.  Sometimes  problem  scenarios  are  used,  but  usually  within  a discrete  subject 
area.  Students  are  sometimes  trained  in  problem  solving  techniques,  but  just  as  often  not. 
The  learning  focus  is  on  finding  answers,  with  the  answers  usually  in  the  information. 

The  solutions  are  "bounded"  and  considered  vital,  with  usually  no  requirement  to  go 
beyond  those  materials.  Problem-based  learning  is  different.  Curricular  content  is 
organized  around  the  problem  scenarios  rather  than  subjects  or  disciplines.  Students  work 
in  teams  or  groups,  and  there  are  no  predetermined  "right"  answers.  The  situation 
presented  is  complex,  and  students  are  expected  to  engage  by  determining  what 
information  they  feel  is  needed  and  what  they  must  learn  or  what  skills  they  need  to  gain 
to  effectively  solve  the  problem  (or  in  a complex  scenario,  problems).  Problem-based 
learning  is  much  more  student  centered;  problem  solving  learning  is  still  more  teacher  or 
professor  centered.  Problem-based  learning  provides  the  context  in  which  students 
explore  a wider  range  of  knowledge  and  skills  or  information.  They  engage  in  linking 
learning  with  their  needs  to  solve  the  problem(s)  and  in  independent  inquiry.  Problem- 
based  learning  requires  flexibility  and  diversity,  in  that  learning  can  be  characterized 
differently  and  implemented  through  a variety  of  methods  and  processes.  It  requires  the 
integration  of  knowledge  and  skills,  disciplines  and  subjects,  and  the  use  of  a wider  range 
of  tools  and  techniques.  The  focus  of  learning  is  the  problem  scenarios,  not  discrete 
disciplines  or  subjects.  Personally,  I like  to  engage  teams  within  the  same  course  in 
different  problem  scenarios  so  they  are  learning  what  has  been  deemed  as  critical,  using 
different  "vehicles"  to  do  so,  instead  of  all  working  on  the  same  problem.  This  taps  into 
student  interest  and  decision  making,  truly  empowering  them  by  giving  authority.  In  my 
opinion,  this  is  also  a way  for  students  to  develop  new  and  use  existing  "common" 
knowledge.  More  importantly,  it  is  a great  method  that  not  only  provides  the  opportunity, 
it  requires  that  "uncommon"  knowledge  (knowledge  that  is  specific  to  each  individual 
student  that  others  with  them  do  not  know)  comes  into  play,  thus  broadening  what  all 
learn  using  each  other  as  knowledge  and  skill  resources.  This  is  critical  to  a team 
environment  because  there  is  no  need  for  a team  unless  it  has  a diverse  range  of 
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knowledge  and  skills.  If  problems  are  designed  appropriately  and  well,  are  appropriately 
complex,  there  is  a need  for  both  "common"  and  "uncommon"  knowledge  as  well  as  the 
"generation"  of  knowledge  by  both  individuals  and  the  team  as  a whole.  Team  learning  is 
critical. 

In  this  type  of  learning  environment,  we  are  creating  student  learning  communities  or  a 
learning  community  with  student  learning  circles.  We  are  using  Senge's  (1990)  learning 
organizational  theory,  mentioned  above.  Students  examine  their  own  knowledge  and 
skills  and  question  each  other  and  their  beliefs  about  their  own  knowledge  in  ways  that 
lectures  and  even  the  problem  solving  methods  simply  cannot  accomplish.  Dewey  (1938) 
established  that  learning  is  not  something  "reliable  and  changeless,"  but  rather  an 

activity,  a process  of  finding  out. . .that  we  are  the  stuff  and  substance  of  the  world 
and  as  such  we  must  work  from  the  middle  of  a situation  in  which  our  most 
reliable  beliefs  are  at  best  imperfect  inadequate  - is  that  we  are  not  spectators,  but 
agents  of  change. . .a  pragmatic  stance  towards  knowledge. . .that  knowledge  was 
bound  up  in  activity,  (as  cited  in  Savin-Baden,  2000,  pp.  2-4) 

Scarborough  (1983)  found  that  students  learned  more  when  engaged  in  activity  rather 
than  just  listening,  that  students  performed  at  a higher  level  when  measured  for  learning 
when  they  (a)  listened,  (b)  listened  and  watched  a demonstration,  and  then  (c)  listened, 
watched  a demonstration,  and  engaged  in  technical  activity.  The  third  method  resulted  in 
higher  learning  levels. 

Problem-based  learning  has  some  of  its  justification  with  the  shift  from  liberal  arts 
education  (education  for  the  sake  of  education)  toward  the  relationship  between 
education  and  industry,  in  which  a set  of  professional  competencies  has  become  the  focus 
of  curricula.  This  was  in  some  ways  contradictory  to  the  type  of  education  needed  for 
students  to  develop  the  capacities  to  operative  effectively  in  society  [and  at  work];  this 
curricula  was  termed  operational  curricula.  Problem-based  learning  bridges  the  two  foci 
of  education  and  curricula,  offering  students  engagement  in  education  that  has  real 
meaning,  valuing  knowledge  both  for  its  own  sake  and  its  market  context.  Learning  with 
complexity  is  where  there  are  no  direct  answers  to  problem  scenarios;  it  is  a process  that 
develops  the  ability  to  learn  as  well  as  solve  the  problem  scenario.  Students  can  link  their 
own  interests  and  perspectives  and  their  own  motivations;  this  occurs  in  a 'real  world' 
context  - the  problem  scenario  and  embedded  problem(s).  The  requirements  of  real  world 
communities  of  practice  can  be  one  foundational  component  of  the  problem  and  scenario. 
Savin-Baden  (2000)  bases  her  argument  for  problem-based  learning  in  higher  education 
as  more  central  than  it  currently  is  on  the  following  themes.  (It  is  important  to  note  that  in 
our  engineering  and  technology  context,  ABET,  the  National  Science  Foundation,  and 
NAIT  expect  learning  to  be  problem  based,  not  just  problem  solving.)  Themes  include 

1 . Problem-based  learning  as  a concept  and  approach  is  often 
misunderstood. . . [and]  has  often  been  confused  with  forms  of  problem  solving 
learning... 

2.  Misunderstandings. . .have  resulted  in  an  underestimation  of  its  value  in  terms 
of  equipping  students  for  a complex  and  changing  professional  life. . . 

3.  There  exist  a number  of  forms  of  problem-based  learning. . . 
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4.  There  is,  as  yet,  little  known  about  what  actually  occurs,  as  it  were,  inside 
problem  based  curricula  in  terms  of  staffs  and  students'  lived  experience  of 
the  curriculum... Second,  key  elements  such  as  learning  context,  learner 
identify  and  'learning  in  relation'  are  rarely  acknowledged  or  discussed  when 
implementing  or  enacting  problem-based  learning. 

5.  Learning  should  be  seen  as  a cyclical  process  in  which  students  make 
'transitions'  through  which  they  develop  increasing... understandings  of 
themselves,  their  context,  and  the  ways  and  situations  in  which  they  learn 
effectively. 

6.  The  full  potential  of  problem-based  learning  will  only  be  achieved  through 

• understanding  and  acknowledging  the  similarities  and  differences  between 
problem-based  and  problem  solving  learning 

• making  the  form  of  problem-based  learning  explicit 

• recognizing  the  impact  of  the  organization  upon  implementation  . . . 

• acknowledging  that  problem-based  learning  can  offer. . .students  the 
opportunity  of  learning  to  'make  sense'  for  themselves,  personally, 
pedagogically  and  interactionally 

• realizing  the  value  and  complexity  of  it  as  an  approach  to  learning  and  the 
ways  in  which  it  can  help  students  to  understand  and  challenge  their 
situations  and  frameworks  by  encouraging  them  to  'learn  with  complexity' 
and  through  ambiguity.  (Savin-Baden,  2000,  pp.  8-9) 

Savin-Baden  (2000)  thoroughly  explores  the  theory  and  research  bases  through  which 
problem-based  learning  emerged  and  the  arguments  related.  She  especially  mentions 
Bouden’s  (1985)  characteristics  in  addition  to  its  student  and  problem  centeredness. 
Problem-based  learning: 

1 . acknowledges  the  base  of  learner  experience 

2.  emphasizes  students’  responsibility  for  their  own  learning 

3.  crosses  disciplines 

4.  intertwines  theory  and  practice 

5.  focuses  on  processes  of  acquiring  knowledge  rather  than  just  the  products  of  such 
processes 

6.  changes  the  professor's  role 

7.  changes  the  focus  of  assessment  from  professor  assessing  student  outcomes  of 
learning  to  student  self  and  peer  assessment 

8.  requires  a focus  on  communication  and  interpersonal  skills;  students  must 
understand  those  requirements  if  they  are  to  be  capable  of  relating  their 
knowledge;  they  need  more  than  their  technical  skills. 

Savin-Baden  (2000)  also  points  out  Barrows’s  (1986)  taxonomy  of  problem-based 
learning  methods,  which  explains  different  meanings  and  uses: 

1 . Lecture-based  cases  - students  are  presented  with  information  through  lectures 
and  then  case  materials  are  used  to  demonstrate  that  information 

2.  Case-based  lectures  - students  are  presented  with  case  histories  or  vignettes  before 
a lecture  that  covers  the  relevant  materials 
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3.  Case  method  - students  are  given  a complete  case  study  that  must  be  researched 
and  prepared  for  discussion  in  the  next  class 

4.  Modified  case-based  - students  are  presented  with  some  information  and  are 
asked  to  decide  on  the  forms  of  action  and  decisions  they  may  make.  Following 
their  conclusions,  they  are  provided  with  more  information  about  the  case 

5.  Problem  based  - students  meet  with  a client  in  some  form  of  simulated  format  that 
allows  for  free  enquiry  to  take  place 

6.  Closed  - loop  problem  based  - an  extension  of  the  problem  based  method,  where 
students  are  asked  to  consider  the  resources  they  used  in  the  process  of  problem 
solving  in  order  to  evaluate  how  they  may  have  reasoned  through  the  problem 
more  effectively. 

Problem-based  learning  is  more  an  educational  strategy  than  a teaching  model.  There  is 
argument  that  for  problem-based  learning  to  be  present,  the  requirements  are 

1.  curriculum  be  organized  around  problems  rather  than  disciplines,  with  an 
integrated  curriculum  and  emphasis  on  cognitive  skills 

2.  conditions  facilitate  problem-based  learning  such  as  small  groups  and  active 
learning 

3.  facilitation  of  outcomes  such  as  the  development  of  lifelong  learning  skills 

Problem-based  learning  is  an  excellent  stage  for  students  to  learn  to  work  in  teams  and 
learn  as  teams,  a primary  requirement  of  business  and  industry  today.  Students  can  enter 
into  debate  where  they  learn  from  each  other.  Learner  identity  is  a key  concept  with 
interaction  between  the  learner  and  learning,  which  forms  a particular  type  of  identity. 
This  includes  but  goes  beyond  learning  styles  of  the  learner.  Students  begin  to  be  aware 
of  how  others  see  them  as  learners  and  realize  a need  to  adapt  their  learning  style  across 
different  learning  contexts.  Learner  identities  evolve  as  students  transcend  subjects  and 
disciplines  and  structures  of  higher  education.  Regarding  problem-based  learning, 
learners  begin  to  challenge  the  framework  rather  than  having  frameworks  imposed  upon 
them.  Technology  can  inhibit  the  development  of  learner  identity  as  it  can  cause  a shift 
back  to  isolation,  individualized  and  solitary,  virtual  learning.  Heppell  and  Ramondt 
(1998)  establish  that  although  there  is  a technological  revolution,  constructivist  models  of 
learning  are  not  yet  part  of  the  revolution.  Problem-based  learning  involves  the  students 
in  "dialogic"  learning  where  understanding  emerges  from  the  learning  environment; 
students  draw  on  their  own  experiences  to  explain  concepts  and  ideas  and  then  use  that  to 
make  sense  or  make  meaning  (Mezirow,  1981). 

Learning  in  context. . . 

is  broader  than  students'  experiences  of  the  curriculum  and  teaching  methods  in 
which  they  are  engaged,  but  a conception  that  acknowledges  the  values  that 
underpin  those  structures,  the  values  that  [professors]  and  students  bring  to  that 
context  and  the  relationships  that  occur  (or  fail  to  occur)  between  students  and 
between  [professors]  and  students.  . . .also  incorporates  the  way  in  which  the 
curriculum  is  situated  within  the  university  and  the  broader  framework  of 
HE, . . . affecting  what  it  means  to  be  a learner  in  those  contexts ....  [and]  not  only 
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comprises  the  formal  curriculum  but  also  the  informal  one  - the  ones  the  students 
create  for  themselves,  (p.  35) 

We  need  to  keep  in  mind  the  difference  between  transferable  skills  and  the  ability  to 
transfer  skills.  We  seek  to  have  students  learn  and  understand  that  knowledge  and  skills 
should  be  transferable  across  contexts,  but  I have  never  heard  anyone  mention  the  issue 
of  the  "ability"  of  transferring  knowledge  and  skills.  Bridges  (1993)  explains  that 
adaptation  across  contexts  requires  the  use  of  meta-skills. 

These  begin  to  look  like  very  sophisticated  personal/intellectual  achievements, 
which  might  explain  why  they  are  not  in  abundant  supply. . .they  look  more  like 
the  kind  of  competence,  capability  or  ability  which  lies  at  the  heart  of  the 
sensitive,  responsive  and  adaptable  exercise  of  professionalism  in  any  sphere  than 
the  atomistic  list  of  'competencies'  towards  which  we  are  sometimes  invited  to 
direct  our  enthusiasm,  (p.  51) 

Learning  in  relation,  “students  learning  with  and  through  others  in  ways  that  help  to  make 
connections  between  their  lives,  with  other  subjects  and  disciplines  and  with  personal 
concerns,  offers  students  particular  kinds  of  learning  opportunities"  (Weil,  1989  as  cited 
in  Savin-Baden,  2000,  p.  36).  This  can  occur,  for  example,  incidentally  in  lectures,  group 
work,  discussion,  debates,  formally  and  informally  as  students  make  an  effort  to  create 
sense  of  their  learning,  their  experiences  and  relationships,  and  their  lives  - making 
connections,  developing  their  own  voice,  and  learning  to  trust  their  own  voice.  Barnett 
(1994)  discusses  this  as  "being  released,"  the  idea  of  developing  one's  own  voice,  which 
goes  beyond  empowerment  and  self  realization  to  develop  the  learners’  identity  by 
constructing  and  articulating  their  own  perspectives  and  defending  those  perspectives 
before  their  peers  and  professors.  These  three  concepts:  "learner  identity,  learning  in 
context,  and  learning  in  relation”  should  establish  the  expectations  and  understanding  of 
what  it  means  to  learn  in  higher  education  if  we  want  to  develop  questioning  practitioners 
in  the  professional  and  very  real  world.  These  concepts  acknowledge  and  value  the  inner 
world  of  the  learner  and  bridge  the  gap  between  formal  and  informal  learning  contexts, 
resulting  in  a different  kind  of  student,  one  who  constructs  himself  and  has  learned  to  use 
his/her  voice. 

Finding  voice  means  that  learners  realize  learning  is  more  than  the  accumulation  of 
knowledge  or  acquisition  of  competencies.  We  moved  away  from  these  curricular  or 
learning  strategies,  as  there  was  a shift  away  from  learning  methods  such  as  group  work 
and  seminars  to  more  individualistic  methods  toward  assessment  lending  itself  to  only 
right  and  wrong  answers  with  corrective  performance  practices  rather  than  more 
challenging  frameworks.  This  process  shifted  to  mass  higher  education,  to  students  as 
consumers,  and  finally  toward  more  technological  environments. 

This  discussion  could  move  on  to  include  "transitional  learning,"  where  the  student 
experiences  shifts  as  a result  of  critical  reflection  - learning  onward  to  explore  the 
"dimensions  of  the  learner  experience,"  personal  stance,  pedagogical  stance,  interactional 
stance,  and  the  domains  of  each  through  which  we  can  trace  the  learner's  development. 
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Finally,  we  need  to  be  eoneemed  about  the  possible  oeeurrenee  of  disjunetion,  referring 
"to  a sense  of  fragmentation  of  part  of,  or  all  of  the  self,  characterized  by  frustration  and 
confusion,  and  a loss  of  sense  of  self. . . [resulting]  in  anger,  frustration,  and  a desire  for 
'right'  answers"  (Savin-Baden,  2000,  p.  87).  A summary  of  problem-based  learning 
models  is  presented  below. 

Table  A. 6. 5:  Models  of  Problem-Based  Learning 
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Constructivism 

The  above  topics  on  knowledge,  transformational  learning,  and  problem-based  learning 
lead  naturally  into  exploration  of  the  "construction  of  knowledge"  or  constructivism, 
since  each  of  the  discussions  holds  inherent  aspects  of  constructivist  theory.  What  is  it? 
"Constructing  knowledge,  not  receiving  it"  (Marlowe  & Page,  1998,  p.  9). 

I tell  you  one  thing,  if  you  learn  it  by  yourself,  if  you  have  to  get  down  and  dig  for 
it,  it  never  leaves  you.  It  stays  here  as  long  as  you  live  because  you  had  to  dig  it 
out  of  the  mud  before  you  learned  it.  (Norton  as  cited  in  Wigginton,  1985, 
Introduction) 
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It  really  means:  constructing,  inventing,  creating,  and  developing  your  own 
knowledge. . .Receiving  it,  getting  it,  and  hearing  it  does  not  equal  learning.  Learning  in 
constructivist  terms  is 

• both  the  process  and  the  result  of  questioning,  interpreting,  and  analyzing 
information 

• using  this  information  and  thinking  process  to  develop,  build,  and  alter  our 
meaning  and  understanding  of  concepts  and  ideas,  and, 

• integrating  current  experiences  with  our  past  experiences  and  what  we 
already  know  about  a given  subject. 

It's  about  thinking  and  analyzing. . .not  accumulating  and  memorizing. . . Rather, 
students  uncover,  discover,  and  reflect  on  content  and  their  conceptions  of  such 
through  inquiry,  investigation,  research,  and  analysis  in  the  context  of  a problem, 
critical  question,  issue,  or  theme. . .Students  gain  and  are  encouraged  to  develop 
through  these  processes  the  ability  to  think  for  themselves,  and  to  think  critically; 
that  is,  to  discriminate  between  the  relevant  and  the  irrelevant,  to  look  at  issues 
from  different  perspectives,  to  interpret  and  analyze  written  and  electronic  data 
[and  other  engineering  data-designs,  etc.]  and  to  "detect  crap". . .It  is  about 
understanding  and  applying,  not  repeating  back. . .It  is  about  being  active,  not 
passive. . .for  as  Dewey  said,  'There  is  no  such  thing  as  genuine  knowledge  and 
fruitful  understanding  except  as  offspring  of  doing.’  (Marlowe  & Page,  1998,  pp. 
9-13) 

Constructivist  learning  theorists  agree  that  constructivist  learning  revolves  around  these 
propositions: 

• students  learn  more  when  they  are  actively  engaged  in  their  own  learning 

• by  investigating  and  discovering  for  themselves,  by  creating  and  re-creating, 
and  by  interacting  with  the  environment,  students  build  their  own  knowledge 
structures 

• learning  actively  leads  to  an  ability  to  think  critically  and  to  solve  problems 

• through  an  active  learning  approach,  students  learn  content  and  process  at  the 
same  time  (Page,  1990) 

In  summary,  educators,  philosophers,  and  psychologists  agree  that  active  learning 
expands  the  brain  (their  work  is  reviewed  in  Marlowe  & Page,  1998).  They  did  not 
always  use  the  term  constructivism;  however,  their  theories  of  learning  engage  students 
in  the  construction  of  their  own  knowledge  while  mentally  active  in  or  on  their 
environment.  Also,  Marlowe  and  Page  review  the  research  results  on  the  use  of  active 
learning  methods  associated  with  constructivism.  The  research,  both  older  and  newer, 
indicates  that  active  learning  methods  are  much  superior  in  achieving  results  when 
measuring  academic,  affective,  and  skill  learning  than  teacher-centered  or  dominated 
methods.  For  example,  the  research  reveals  that  active  learning  and  constructivism  result 
in  greater  exchange  of  ideas  between  students.  Their  participation  greatly  increases.  They 
learn  to  develop  hypotheses;  use,  interpret,  and  apply  evidence;  and  perceive  knowledge 
as  more  tentative,  not  as  absolute.  Students  gain  in  creativity  and  intelligence,  increase 
language  skills,  and  use  math  more  effectively.  Other  students  reported  students  were  less 


106 


bored.  When  engaging  students  in  cooperative  learning  alongside  constructivism,  there 
was  a highly  superior  level  of  learning  difference  with  the  cooperative  learning  groups. 

Brooks  and  Brooks  (1993)  offer  a set  of  descriptors  for  constructivist  teaching  behaviors 
as  a framework  from  which  to  operate: 

1.  Constructivist  teachers  encourage  and  accept  student  autonomy  and  initiative. 

2.  Constructivist  teachers  use  raw  data  and  primary  sources,  along  with 
manipulative,  interactive,  and  physical  materials. 

3.  When  framing  tasks,  constructivist  teachers  use  cognitive  terminology  such  as 
"classify,"  "analyze,"  "predict,"  and  "create." 

4.  Constructivist  teachers  allow  student  responses  to  drive  lessons,  shift 
instructional  strategies,  and  alter  content. 

5.  Constructivist  teachers  inquire  about  students'  understandings  of  concepts 
before  sharing  their  own  understandings  of  those  concepts. 

6.  Constructivist  teachers  encourage  students  to  engage  in  dialogue,  both  with 
the  teacher  and  with  one  another. 

7.  Constructive  teachers  encourage  student  inquiry  by  asking  thoughtful,  open- 
ended  questions  and  encouraging  students  to  ask  questions  of  each  other. 

8.  Constructivist  teachers  seek  elaboration  of  students'  initial  responses. 

9.  Constructivist  teachers  engage  students  in  experiences  that  might  engender 
contradictions  to  their  initial  hypotheses  and  then  encourage  discussion. 

10.  Constructivist  teachers  allow  wait  time  after  posing  questions. 

1 1 . Constructivist  teachers  provide  time  for  students  to  construct  relationships  and 
create  metaphors. 

12.  Constructivist  teachers  nurture  students'  natural  curiosity  through  frequent  use 
of  the  learning  cycle  method,  (pp.  103-116) 

Atkin  and  Karplus  (1962)  describe  that  method  as  providing  students  with  an  open-ended 
opportunity  for  students  to  interact  with  purposefully  selected  materials  from  which 
students  generate  questions  and  hypotheses  - known  as  discovery.  Then  the  teacher 
introduces  the  concept  introduction  to  help  focus  students'  questions,  to  frame  lab 
experiences,  etc.,  and  the  students  engage  in  concept  application  (when  students  work  on 
new  problems).  This  is  often  in  opposition  to  how  curricula,  syllabi,  or  texts  are  designed, 
as  they  usually  introduce  the  concept  first,  move  to  application,  and  discovery  takes  place 
afterwards. 
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Table  A. 6. 6:  School  Environments 


Figure  2.1 

A Look  at  School  Environments 


Traditional  Classrooms 

Curriculum  is  presented  part 
to  whole,  with  emphasis  on 
basic  skills. 

Strict  adherence  to  fixed 
curriculum  is  highly  valued. 

Curricular  activities  rely 
heavily  on  textbooks  and 
workbooks. 


Students  are  viewed  as 
“blank  slates"  onto  which 
information  is  etched 
by  the  teacher. 

Teachers  generally  behave 
in  a didactic  manner, 
disseminating  information 
to  students. 

Teachers  seek  the  correct 
answer  to  validate  student 
learning. 


Assessment  of  student 
learning  is  viewed  as 
separate  from  teaching 
and  occurs  almost 
entirely  through  testing. 


Students  primarily  work 
alone. 


Constructivist  Classrooms 

Curriculum  is  presented 
whole  to  part  with 
emphasis  on  big  concepts. 

Pursuit  of  student 
questions  is  highly  valued. 

Curricular  activities 
rely  heavily  on  primary 
sources  of  data  and 
manipulative  materials. 

Students  are  viewed  as 
thinkers  with  emerging 
theories  about  the 
world. 

Teachers  generally  behave 
in  an  interactive  manner, 
mediating  the  environment  for 
students. 

Teachers  serek  the 
students’  points  of  view 
in  order  to  understand 
students’  present 
conceptions  for  use  in 
subsequent  lessons. 

Assessment  of  student 
learning  is  interwoven 
with  teaching  and  occurs 
through  teacher 
observations  of  students 
at  work  and  through 
student  exhibitions  and 
portfolios. 

Students  primarily  work 
in  groups. 


(Brooks  & Brooks,  1993) 
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Active  Learning 


Much  of  the  above  discussion  about  knowledge,  its  transformation,  learning,  critical 
thinking  and  constructivism  (either  directly  or  indirectly)  mentions  or  includes  active 
learning  as  a requirement.  Meyers  and  Jones  (1993),  Twining  (1991),  and  Sutherland  and 
Bonwell  (1996)  all  strongly  support  active  learning  strategies.  Dale's  Cone  of  Learning 
(2001)  has  been  a standard  or  model  supporting  more  active  learning  and  its  benefits  to 
learners. 

Figure  A. 6. 21:  Dale’s  Cone  of  Learning 


After  2 Weeks  We 
Remember  This 


Nature  of  Involvement 


10  % of  what  we  READ 

/ Reading  \ Verbal  Receiving 

20%  of  what  we  HEAR 

/ Hearing  Word^ 

30  % of  what  we  SEE 

/ Looking  at  PicturesX 

50  % of  what  we 
HEAR  & SEE  / 

/ Watching  a Movie  \ 

' Looking  at  an  Exhibit  \ 
Watching  a Demonstration 
Seeing  it  Done  on  Eocation 

Visual  Receiving 

70  % of  what  we  j 
SAY  / 

Participating  in  a Discussion 
Giving  a Talk 

\ Receiving/ 

\ Participating 

90  % of  what 
we  SAY  & 
DO 


Doing  in  a Dramatic  Presentation 
Simulating  the  Real  Experience 
Doing  the  Real  Thing 


Doing 


Dale’s  Cone  of  Learning  (1969) 


(Dale,  1969,  n.p.) 

Passive  learning  occurs  when  teachers  do  most  of  the  work  and  students  are  passive  most 
of  the  time.  Conversely,  active  learning  is  when  students  have  the  opportunity  to  talk  as 
well  as  listen,  read,  write,  reflect  (research,  explore,  inquire,  design,  build,  test,  analyze, 
evaluate,  and  more).  Important  to  note  is  that  students  will  learn  their  course  content 
through  problem  solving  and  PBL,  small  group  activities,  simulations,  case  studies,  and 
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more  - all  very  active  types  of  learning  that  engage  them  in  building  knowledge  and 
skills.  Two  assumptions  are  basic  to  active  learning:  "1)  that  learning  is  by  nature  an 
active  endeavor  and  2)  that  different  people  learn  in  different  ways"  (Dale,  2001,  p.  xi). 
Teaching  corollaries  follow:  "1)  students  learn  best  when  applying  subject  matter,  in 
other  words,  learning  by  doing  and  2)  teachers  who  rely  exclusively  on  any  one  teaching 
approach  often  fail  to  get  through  to  significant  numbers  of  students"  (p.  xi).  The  result  is 
dissatisfaction  on  the  part  of  both  the  teacher  and  students.  Using  active  learning 
increases  the  odds  that  students  leave  the  classroom  with  more  than  notes  on  facts. 
(Meyers  & Jones,  1993,  p.  xi).  Bransford's  (1979)  research  demonstrated  that  when 
information  is  rehearsed  or  used  to  solve  problems,  students  are  more  likely  to  retain  it. 
Activity  fosters  better  retention  and,  more  importantly,  leads  to  students'  expanding  their 
thinking  abilities.  The  theory  behind  this  is  that  semantics  (memory  for  facts)  and 
episodic  memory  (memory  for  events)  interact  to  bring  about  long-term  memory  of  what 
is  learned  because  of  the  action  or  active  learning  through  events.  This  means  that 
teaching  and  learning  become  more  cooperative  in  nature  and  that  students  have  a greater 
role  in  learning.  Combined  with  immediate  feedback,  a characteristic  of  active  learning, 
students  understand  their  contributions  better  and  see  their  value.  Certainly  today,  a 
critical  factor  is  using  learning  strategies  that  engage  women  and  a culturally  diverse 
learning  group  of  students.  An  engineering  and  technology  college  such  as  ours  is  highly 
diverse  in  culture,  gender,  and  ages.  Active  learning  involves  students  in  cooperation 
where  isolation  and  competitiveness  existed  before;  therefore,  the  learning  environment 
is  appropriate  for  a wider  range  of  student  perspectives  and  learning  strategies.  As 
Pintrich  (1988)  notes,  "little  has  been  done  to  translate  the  research  on  learning  and 
cognition  'into  directly  applicable  information  relevant  to. . .classroom  practice'"  (p.  72). 
Remember,  there  is  value  to  lectures,  IF  they  are  good  and  well  delivered.  Lectures 

dramatize  the  creation  of  knowledge. . .and. . .interpret  knowledge  for  listeners; 
students  learn  from  inspired  lectures. . .but  a steady  diet  of  lecturing  leads  to 
intellectual  anemia. . .lectures  are  more  engaging  when  . . .punctual  with  brief 
active  learning  exercises  that  enhance  retention  and  application. . . [so  students 
can]  use  [the  knowledge  presented]  in  class  and  apply  it  to  their  everyday  lives. 
(Corder,  1991,  p.  B2) 

The  "empty  vessel"  or  "additive  theory,"  where  knowledge  is  imparted  to  fill  the  students' 
vessels  or  to  add  to  what  they  might  already  know  does  not  accurately  reflect  how 
students  come  to  us.  They  come  with  their  own  knowledge  frameworks,  as  discussed 
above.  Their  perceptual  frameworks  are  very  much  intact,  sometimes  even  entrenched 
(the  difficult  student),  and  we  know  from  Kolb  (1984)  and  Briggs-Myers  (1980)  that 
students  learn  in  different  ways.  The  amount  of  information  and  new  insights  has 
exploded  to  further  challenge  us  in  our  disciplines,  leading  to  the  point  of  view  that  we 
simply  cannot  take  the  teaching/learning  approach  that  we  are  passing  on  a "static"  or 
"known"  body  of  discrete  knowledge  (also  mentioned  above).  Cross  (1991)  says: 
"Learning  is  not  so  much  an  additive  process,  with  new  learning  simply  piling  up  on  top 
of  existing  knowledge,  as  it  is  an  active,  dynamic  process  in  which  the  connections  are 
constantly  changing  and  the  structure  reformatted"  (p.  9).  Therefore  teaching  has  and 
must  continue  to  change... from  "transmission"  to  "dialogue"  or  "communication";  in 
other  words,  from  passive  to  active  (Tiberius,  1986).  Also  Palmer  (1987)  views  teaching 
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and  learning  "as  a shift  in  focus  from  the  individual  as  the  'agent  of  knowing'  to  learning 
as  a communal  act"  (pp.  20).  Rather  than  the  more  transmissive  approach  - or 
transactional  approach  - where  the  "ignorant"  student  is  the  recipient  of  the  "truth"  from 
the  teacher,  learning  is  dynamic  and  an  interchange  between  the  teacher  and  student,  a 
cooperative  of  inquiry  where  a communal  dialogue  takes  place. 

Furthermore,  Hutchins  (1990)  feels  that  what  matters  “is  not  just  what  students  know  but 
what  they  can  do  with  what  they  know.  What's  at  stake  is  the  capacity  to  perform,  to  put 
what  one  knows  into  practice.”  This  certainly  fits  into  our  purpose  of  performance 
assessment  as  a part  of  that  picture.  As  tests  are  only  indicators  of  what  students  might  be 
able  to  do,  a logical  part  of  any  student  assessment  plan  should  be  performance 
assessment.  Teachers  are  beginning  to  realize  a new  role,  that  of  the  facilitator  of 
learning,  where  once  again  in  my  opinion,  the  burden  of  learning  shifts  back  to  the 
student.  We  should  create  opportunities  for  students  not  only  to  construct  knowledge  as 
mentioned  above,  but  to  give  birth  to  their  own  ideas,  to  support  student  thinking,  and  to 
provide  the  context  to  deepen  their  inquiry  and  thus  their  understanding  (Belenky,  et  ah, 
1986).  Students  should  be  active  in  the  teaching/learning  process.  So  what  really  is  active 
learning  and  how  do  we  begin  to  transform  our  teaching  strategy  from  lectures  to  more 
active  learning  strategies?  First,  let  us  remind  ourselves  what  lectures  do  IF  well  planned, 
well  organized,  well  delivered,  high  content  quality,  and  where  a pre-organization 
teaching  strategy  is  used  to  "set  up"  for  the  lecture.  This  description  of  a good  lecture 
alone  would  make  many  realize  that  their  lectures  have  not  yet  achieved  this  level  of 
quality.  However,  let  us  assume  for  the  moment  that  our  lectures  are  very  good.  Cashin 
(1985)  feels  that  they 

• Provide  information  that  is  new,  based  on  original  research,  and  generally 
not  found  in  textbooks  and  other  printed  sources  (not  true  for  those  who 
lecture  from  their  textbooks. 

• Highlight  similarities  and  differences  between  key  concepts 

• Help  communicate  the  enthusiasm  of  teachers  for  their  subjects 

• Model  how  a particular  discipline  deals  with  questions  of  evidence,  critical 

analysis,  problem  solving,  and  the  like 

• Dramatize  important  concepts  and  share  personal  insights 

• Organize  subject  matter  in  a way  that  is  best  suited  to  a particular 
class  and  course  objectives  (p.  10) 

One  of  the  primary  arguments  teachers  and  professors  use  as  justification  for  lecturing  is 
that  students  need  some  background  information,  theories,  concepts,  methods,  etc.  before 
they  can  learn  on  their  own  or  become  active  and  participants  with  each  other.  Or  if 
lectures  are  not  used,  students  will  not  have  a notebook  of  concepts,  facts,  etc.  at  the  end 
of  the  course.  These  are  legitimate  arguments,  but  teachers  and  professors  can  cover  all 
that  content  using  active  learning  strategies  in  complement  with  or  in  place  of  lectures. 
However,  active  learning  strategies  are  no  more  successful  than  poorly  executed  lectures 
if  not  planned  and  executed  well.  One  should  not  consider  the  choice  as  either  lecture 
"or"  active  learning.  Usually  there  is  too  much  lecture,  rather  than  an  appropriate  amount 
that  leads  students  into  activity  oriented  learning  beyond  the  lecture.  The  focus  should  be 
to  "uncover"  the  knowledge  with  our  students,  provide  opportunities  for  them  to  dig  into 
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the  subject  matter  to  "make  sense"  of  it  and  to  change  or  expand  their  knowledge 
frameworks  (Dawkins,  1972).  So  to  consider  this  seriously,  what  teachers  would  rather 
not  know  is: 

• While  teachers  are  lecturing,  students  are  not  attending  to  what  is  being 
said  40%  of  the  time.  (Pollio,  1984,  p.  1 1) 

• In  the  first  ten  minutes  of  lecture,  students  retain  70  percent  of  the 
information;  in  the  last  ten  minutes,  20  percent.  (McKeachie,  1986,  p.  72) 

• Students  lose  their  initial  interest,  and  attention  levels  continue  to  drop, 
as  a lecture  proceeds.  (Vemer  & Dickinson,  1967,  pp.  90-91) 

• Four  months  after  taking  an  introductory  psychology  course,  students 
knew  only  8%  more  than  a control  group  who  had  never  taken  the  course. 
(Rickard,  Rogers,  Ellis,  & Beidleman,  1988,  pp.  151-152) 

Do  not  throw  lectures  out  totally;  use  them  appropriately  with  a wider  range  of  teaching 
and  learning  strategies.  Realize,  however,  that  lectures  many  consider  good  are  really  not 
good  at  all,  based  upon  content  choice  and  other  factors.  Also  lecture  based  courses  send 
the  message  that  we  as  professors  feel  we  know  it  all.  Of  course,  we  do  not  and  never 
will.  Perhaps  we  should  be  more  open  and  honest  about  that  as  well.  And  when 
professors  lecture  only  from  the  textbook,  then  that  alone  is  great  evidence  that  they  must 
not  know  anything  beyond  the  textbook.  Many  professors  find  that  active  learning 
strategies  are  more  natural  to  their  own  unique  styles,  personalities,  and  preferences  than 
lectures,  thus  their  teaching  improves  because  of  a better  match  of  strategy  and 
individual.  Some  struggle  with  both,  and  yet  others  struggle  to  learn  to  use  active  learning 
strategies  comfortably  but  are  glad  when  they  do,  as  they  begin  to  see  changes  in  their 
students  and  learning.  Angelo  and  Cross  (1993)  offer  teaching  assessment  strategies  in 
their  book  on  assessment.  They  suggest  asking  students  for  feedback  on  strategies  as 
professors  strive  to  increase  their  repertoire  of  strategies.  Active  learning  strategies  take 
more  time  than  lectures;  this  is  also  where  teachers  or  professors  begin  to  argue  they 
cannot  use  them  because  they  will  not  be  able  to  cover  the  content.  However,  as  one 
professor  noted,  I was  not  covering  the  last  six  chapters  very  well  anyway  and  students 
were  not  doing  well  on  the  tested  content,  so  in  fact,  I was  not  accomplishing  much; 
therefore,  why  not  use  that  time  for  more  active  learning  where  students  come  out 
knowing  and  retracting  much  more. 

As  a result  of  active  learning,  students  will  begin  to  cover  more  content  than  expected 
and  concepts,  theories,  and  information  begin  to  cluster  and  take  on  a life  of  their  own. 
Sometimes  much  more  will  be  covered  as  students  engage  and  seek  the  information  they 
need  to  accomplish  their  learning  tasks  or  activities.  Linearity  disappears  when  students 
begin  to  “integrate”  across  knowledge  and  skills  they  are  learning.  Meyers  and  Jones 
(1993)  make  an  important  statement  of  belief. 

encouraging  students  to  be  self-directed  and  collaborative,  critically  reflective, 
politically  savvy,  empathic,  and  fair  minded,  as  well  as  competent  in  the  skills 
that  are  essential  to  meaningful  lives  and  careers.  We  are  confident  that  despite 
their  sometimes  passive,  if  not  apathetic,  exteriors,  most  students  are  capable  of 
acquiring  those  abilities  because  deep  inside  remains  a desire  to  explore  and  to 
learn.  Such  students  who  learn  to  take  responsibility  for  their  own  learning  will 
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help  make  our  society  more  democratic;  and  a better  place  for  everyone.  Finally, 
active  learning  helps  prepare  our  students  to  be  self-directed,  lifelong  learners  - 
an  ability  they  will  all  need  in  a society  where  individuals  change  jobs  numerous 
times  in  their  working  years  and  have  extended  leisure  time  after  retirement. . .We 
claim  no  originality  here,  as  the  sages  and  experts  we  cite  and  depend  on 
[throughout  the  book]  attest.  But  we  want  to  be  clear  that  our  personal 
commitment  to  active  learning  goes  deeper  than  its  efficiency  as  a teaching  tool 
and  that  it  is  essential  to  our  vision  of  what  an  educated  person  should  be.  ..Those 
who  agree  that  teaching  is  more  than  transmitting  information. . . will  agree  that 
making  room. . .for  a variety  of  active  learning  strategies  is  a natural  step. . .Those 
who  accept  that  premise  that  different  students  learn  in  different  ways  and  are 
frustrated  with  not  teaching  their  students  by  traditional  teaching  will  find  that 
active-learning  strategies  not  only  enliven  their  classroom  but  significantly 
improve  their  students'  thinking  and  learning  capabilities,  (p.  17) 

Active  learning  can  be  described  as  three  interrelated  factors:  basic  elements,  learning 
strategies,  and  teaching  resources.  Basic  elements  are  what  students  do  (e.g.,  talk  and 
listen,  reading,  writing,  and  reflecting).  Learning  strategies  are  group  work,  case  studies, 
etc.  Teaching  resources  are  speakers,  homework  assignments,  etc.  There  can  be  an 
interrelationship  within  the  sub  factors  and  also  between  the  primary  factors,  elements, 
strategies,  and  resources. 

Figure  A. 6. 22  : Structure  of  Active  Learning 

Figure  2.1.  Structure  of  Active  Lei^rning. 


Elements 

talking  and  listening 
writing 
reading 
reflecting 

i 


Learning  Strategies 

small  groups  cooperative  work  case  studies  simulations 
discussion  teaching  problem  solving  journal  writing 


t 


Teaching  Resources 

readings  homework  assignments  outside  speakers 
teaching  technology  prepared  educational  materials  commercial  and  educational  television 


(Piaget,  1976,  p.  20) 
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Remember  their  assumptions:  "1)  learning  is  by  its  very  nature  an  active  process  and  2) 
different  people  learn  in  different  ways"  (p.  20).  They  agree  that  learners  construct  their 
own  knowledge,  as  discussed  above,  using  Piaget's  concept  of  mental  structures. 

Children  [students]  do  not  receive  knowledge  passively  but  rather  discover  and 
construct  knowledge  through  activities.  As  children  [students]  interact  with  their 
psychological  and  physical  environments,  they  begin  to  form. . .structures  of 
thought.  These  structures  help  to  organize  the  child's  [student’s]  experience  and 
direct  future  interactions”  (Piaget,  1976,  p.  119). 

Meyers  and  Jones  (1993)  agree  with  him  about  a basic  principal:  "students,  no  matter 
what  their  age,  need  opportunities  to  engage  in  activities  - with  teachers,  fellow  students, 
and  materials  - that  help  them  create  their  own  mental  structures  and  test  them,  thus 
making  better  sense  of  the  world  around  them”  (p.  21).  Therefore,  they  identify  the  four 
key  elements  of  active  learning  used  to  create  new  mental  structures  as  talking  and 
listening,  reading,  writing,  and  reflecting.  This  involves  cognitive  activities  so  students 
will  appropriate  new  knowledge  - IF  activities  are  well  structured,  planned,  and  guided 
by  teachers.  They  clearly  describe  each  of  these  elements  and  how  to  do  them  well,  also 
describing  them  when  poorly  executed.  The  writing  section  is  especially  helpful  in  that 
they  address  clarity  of  instructions.  For  example,  what  does  a teacher  or  professor  mean 
when  they  tell  students  to  analyze  something?  What  does  the  term  "analyze"  convey  to 
the  student?  That  certainly  depends  upon  their  existing  mental  structure  unless  the 
professor  defines  it.  Meyers  and  Jones  refer  to  Fulwiler’s  (1987)  help  typology  of  words 
for  written  assignments: 

• Analyze:  Take  apart  and  look  at  something  closely. 

• Compare:  Look  for  similarities  and  differences;  stress  similarities. 

• Contrast:  Look  for  differences  and  similarities;  stress  differences. 

• Define:  Explain  exactly  what  something  means. 

• Describe:  Show  what  something  looks  like,  including  physical  features. 

• Evaluate:  Make  a value  judgment  according  to  some  criteria  (which  it 
would  be  wise  to  make  clear). 

• Justify:  Argue  in  support  of  something;  to  find  positive  reasons. 

• Prove:  Demonstrate  correctness  by  use  of  logic,  fact,  or  example. 

• Summarize:  Pull  together  the  main  points. 

• Synthesize:  Combine  or  pull  together  pieces  or  concepts,  (pp.  117-118) 

These  definitions  are  especially  helpful  in  consideration  of  our  use  of  Bloom's  Revised 
Taxonomy  of  Eearning  (Anderson  & Krathwohl,  2001).  They  refer  to  the  authors  used 
herein,  Mezirow  et  al.  (1990).  They  also  rely  on  Eawson  and  Renner's  (1975)  work 
mentioned  herein  on  stress  disequilibrium  and  equilibrium  as  important  processes  when 
forming  new  mental  structures.  In  other  words,  when  knowledge  fits  into  existing  mental 
structures  or  knowledge  frameworks,  then  there  is  equilibrium.  However,  when 
experiences  and  new  knowledge  do  not  fit  with  the  existing  structures  or  framework,  then 
disequilibrium  occurs  - a challenging  situation  where  students,  or  we,  must  construct  new 
structures  or  change  or  expand  existing  frameworks  to  incorporate  the  new  knowledge. 
This  requires  reflection,  or  quiet  time,  so  students,  or  we,  can  integrate  the  new 
knowledge.  Thus,  it  is  important  to  build  these  reflection  times  into  learning  (e.g.. 
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through  silence,  journals  - for  me,  concept  mapping).  Meyers  and  Jones  (1993)  go  much 
deeper  into  the  requirements  for  well  designed  active  learning.  They  thoroughly  discuss 
the  course  objectives,  syllabus,  teaching  and  learning  space,  classroom  tone  (positive), 
the  need  to  know  one's  students,  and  more.  These  are  not  new  issues  for  me,  as  I have 
mentioned  before.  To  me  these  topics  are  often  dismissed  by  teachers  or  professors;  they 
feel  that  it  is  not  their  responsibility  to  address  much  more  than  the  syllabus.  I feel  much 
differently.  I have  seen  the  climate  and  environment  change  the  students'  responsiveness 
and  desire  to  be  in  the  course.  (More  on  this  later.)  They  go  in  depth  into  several  active 
learning  strategies,  small  group  work,  cooperative  student  projects,  simulations,  case 
studies,  all  of  which  I have  used  very,  very  successfully  with  students.  They  simply  work 
when  well  planned  and  with  guided  execution.  However,  there  is  much  to  learn  about 
each  one.  (See  these  and  other  authors  for  details.)  In  discussing  teaching  resources,  they 
present  print  sources  and  speakers  with  great  detail  on  what  to  consider  as  effective. 
There  is  an  equally  good  section  on  technology  (although  much  has  changed  since  the 
early  nineties),  and  more  reading  on  selected  resources  - an  annotated  bibliography. 

Bloom’s  Taxonomy  of  Learning. 

Sutherland  and  Bonwell  (1996),  along  with  Twining  (1991),  provide  great  resources  on 
active  learning  using  Bloom's  Taxonomy  as  a conceptual  foundation  of  argument  and 
then  moving  on  to  the  structural  or  critical  element  from  the  perspective  of  established 
research  mentioned  above.  They  discuss  the  lecture,  managing  time,  purposeful  study, 
reading  and  taking  notes  from  textbooks,  vocabulary  and  concept  development,  memory, 
taking  tests,  writing  as  active  learning,  electronic  tools,  cooperative  learning,  and  wrap 
their  book  with  a discussion  of  issues. 


Figure  A. 6. 23:  Bloom’s  Taxonomy  of  Learning 


(Bloom,  1956,  n.p.) 
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Anderson  and  Krathwohl  (2001)  present  a revised  Bloom’s  Taxonomy,  both  knowledge 
and  eognitive  proeess  dimensions.  Their  book  is  very  thorough  and  a good  resouree  for 
faculty  who  desire  to  use  Bloom’s  Taxonomy  more  deeply  and  meaningfully. 

Figure  A. 6. 24:  Bloom’s  Revised  Taxonomy  of  Learning 


Bloom ’s  Revised  Taxonomy  of  Learning 

Anderson  & Krathwohl,  (2001) 


The 

Knowledge 

Dimension 

The  Cognitive  Process  Dimension 

1 . Remember 

2.  Understand 

3.  Apply 

4.  Analyze 

5.  Evaluate 

6.  Create 

A. 

Factual 

Knowledge 

B. 

Conceptual 

Knowledge 

C. 

Procedural 

Knowledge 

D. 

Meta-Cognitive 

Knowledge 

(Anderson  & Krathwohl,  2001,  p.  100) 

Multiple  Intelligences 

When  discussing  knowledge,  it  is  important  to  take  note  of  several  other  perspectives  on 
knowledge  and  learning.  Gardner's  (1983/1993)  work  has  been  very  important  in  that  he 
identified  six  different  ways  to  learn,  which  had  implications  for  styles  of  learning.  In 
Frames  of  Mind,  The  Theory  of  Multiple  Intelligences,  a seminal  work,  he  discusses  the 
importance  of  considering  intelligence  as  "expressed  in  the  context  of  specific  tasks, 
domains,  and  disciplines"  (p.  xvi).  The  most  significant  advancement  in  multiple 
intelligence  theory  is  that  of  distinguishing  between  intelligences,  domains,  and  fields. 
The  original  work  did  not  make  these  distinctions.  There  is  now  a taxonomy 
(Csikszentmihalyi  & Csikszentmihalyi,  1988;  Feldman,  1980,  1986;  Gardner). 

• Intelligence  (and  cognition)  - intellectual  strengths  or  competences . . .certain 
forms  of  cognition. . .a  neural  organization  that  proves  hospitable  to  the  notion 
of  different  modes  of  information  processing  (p.  59) 

• Intelligences  (at  the  individual  level)  - this  is  where  one  or  more  human 
intelligences  exist  or  "human  intellectual  proclivities,"  neurobiological 
capabilities  (p.  xvi).  The  intelligences  identified  by  Gardner  are  linguistic, 
musical,  logical-mathematical,  spatial,  bodily-kinesthetic,  and  personal.  He 
also  suggests  that  there  will  never  be  an  "irrefutable  and  universally  accepted 
list.  (p.  60) 
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Table  A.6.7:  Types  of  Human  Intelligences 


TABLE  5.1  Types  of  Human  Intelligences 


Intelligeuce 

Learner's  Strengths 

Intrapersonal 

Ability  to  know  self;  ability^  to  understand 
one's  own  strengths/ weaknesses  and 
motivations 

Interpersonal 

Ability  to  know  others,  ability  to  "read" 
social  and/or  political  situations;  ability 
to  influence  others;  ability  to  load  and/or 
care  for  others — to  be  sensitive  to  needs 
of  others 

Hod  ily- kinesthetic 

Ability  to  control  the  movement  of  one's 
body:  ability  to  move  in  graceful,  highly 
coordinated  fashion 

Musical 

Ability  to  produce,  write,  and/or  appreci- 
ate music 

Spiihal 

Ability  to  shape,  perceive,  design,  and/ or 
conceive  visual-spatial  information; 
ability  to  remember  visual  information 

l-ogical-nii»tliemat  ical 

Ability  to  manij.iulate  numbers  and  sym- 
bolic inform, ition;  ability  to  draw  logical 
conclusions;  ability  to  think  abstractly 

Verba  I- linguistic 

Ability  to  manipulate,  create,  and  appre- 
ciate the  rliythms  of  language;  ability  to 
speak,  read,  and/or  write  fluently 

(Gardner,  1993,  p.  62) 

• Domain  (or  discipline)  - part  of  the  culture  within  which  one  is  born,  where 
disciplines,  crafts,  and  other  pursuits  can  be  identified  and  where  one  becomes 
acculturated  at  varying  levels  of  competency.  Levels  of  competency  can  be 
assessed.  These  might  be  perceived  as  types  or  areas  of  expertise. 

A relationship  exists  between  intelligences  and  domains  (e.g.,  a person  exhibiting  musical 
intelligence  will  probably  be  attracted  to  the  domain  of  music),  but  other  intelligences  are 
necessary  to  achieve  musical  performances  (e.g.,  bodily-kinesthetic,  personal,  etc.).  Most 
domains  require  multiple  intelligence  proficiencies,  while  any  one  intelligence  can  be 
used  across  a wide  variety  of  available  domains. 

• Field  - a sociological  construct  - includes  the  people,  institutions,  award 
mechanisms,  etc.  that  render  judgments  about  the  qualities  of  individual 
performances.  To  the  extent  that  one  is  judged  competent  by  the  field,  one  is 
likely  to  become  a successful  practitioner;  on  the  other  hand,  should  the  field 
prove  incapable  of  judging  work,  or  should  it  judge  the  work  as  being  deficient, 
then  one's  opportunity  for  achievement  will  be  radically  curtailed. 
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When  considering  intelligences,  one  must  first  consider  prerequisites  for  an  intelligence 
and  its  inclusion  in  the  list.  Along  with  the  criteria  to  determine  an  intelligence,  there 
must  be  a counter  set  of  criteria  or  factors  that  would  suggest  a competence  or  potential 
intelligence  is  not  appropriate  as  an  intelligence.  Furthermore,  Gardner  (1983/1993) 
clearly  states  that  when  using  the  intelligences,  it  is  important  to  remember  that  they  are 
fiction,  possibly  useful  fictions,  for  discussing  processes  and  abilities,  "separately  defined 
and  described  strictly  in  order  to  illuminate  scientific  issues  and  to  tackle  pressing 
practical  problems"  (p.  70).  He  further  describes  each  intelligence  in  depth.  Additionally, 
he  discusses  educational  processes  and  the  application  of  the  intelligences  as  ways  of 
learning  (e.g.,  observational  learning  situated  in  a particular  context,  instruction  outside 
of  the  context  in  which  the  knowledge  or  skill  is  practiced)  occur.  Also  there  are  three 
primary  means  or  media  of  transmitting  learning:  direct  forms  with  verbal  interaction 
using  articulated  symbol  systems  (e.g.,  language  or  mathematics,  books,  charts,  maps, 
computers,  etc.  or  combinations  of  these  media).  Each  media  may  differ  in  the  kind  of 
intelligence  necessary.  Second,  there  are  sites  or  particular  loci  where  learning  can  take 
place.  This  is  where  learners  are  placed  for  learning  on  site  in  a community  of  practice  in 
the  "real  world"  or  in  an  institution  of  learning.  The  third  variable  is  the  agent,  or  person 
in  charge  of  learning  (e.g.,  teachers,  professors,  trainers,  peers,  etc.).  It  is  his/her  job  to 
teach  someone.  Finally,  Gardner  discusses  the  "general  context  in  which  learning  takes 
place"  (p.  336):  the  cultural  context  where  there  may  be  great  value  differences  towards 
intelligences  depending  upon  the  cultural  context.  A technological  society  or  culture  has 
different  values  than  a religious  cultural  context.  There  is  a wide  range  of  educational 
settings  and  experiences  (p.  339).  (See  the  framework  below  to  better  understand 
educational  and  cultural  settings.) 
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Table  A. 6. 8:  Framework  for  Analyzing  Educational 
Processes 


The  framework  for  Analozin;^  Educational  Proctssci  Applied  to  Three  Cultural  Seltmp 


Tvpf  Learning 

Speeialiied  Skill 
III  Shmliieraie  Society 

liffmi  v III  Traditimial 
Retij/ioui  School 

Sf/C'i'i/u*  C«r»it  ulum 
tH  Wudefn  SffnUr  Sihool 

tramples  Menltoned 
m Chapter  O 

Puluvvat  sailing 
Yugoslavian  oral  verse 

Koranic  school 
Hindu  gurukula 
Hebrew  chedcr 
Medieval  vathcdral  school 

Elementary  and  secondary 
schools  in  Europe.  N'orlh  .^merl 
ca  and  lapan.  Programming  on 
mn  rocompuler 

lntelli|5v‘nces 

l-inguKtcc  musiciil  (nrj! 

Sp.itul  (sailing) 

Boililv-kiiieMhptiC 

Jnirrpprsonal 

Linguistic 
Inierperxinal 
Lcigical-tnnihematical 
(among  advanceil  Mu- 
dellls) 

Logical-mathematual 

Inirapersonal 

Linguistic  (less  emphasised) 

til 

Mosllv  unmediated  (direct 
tihvervahop) 

Some  oral  lingumtir  instruc* 
(ion 

Oral  verso  or  books 

Gre.ll  variety,  including  books 
charts,  computers,  hims,  etc 

t tK  us  1 {Mrmng 

On  slip 

Separate  building  or  inside 
religious  building 

Separate  building 

Some  learning  can  be  done  in  a 

private  borne  or  study 

Wh\»  Transmii 

Skilled  elders,  (vpically  rrla* 
nve*. 

Individuals  trained  in  lit- 
oracy  and  argument,  high 
moral  comporlment  ex- 
p»ntrd  status  high  except 
at  entry  level  positions 

Individuals  with  training  in  edu 
cation  ai  lower  level,  individuatJi 
with  specialized  training  ,«i  high- 
er  levels,  moral  caliber  not 
germane 

< mI  Contest  Df 

1 

Mini  indivnlujls  shjrr  soirn- 
ha.sif  skills,  including  sailing 
a leiv  may  be  experts 

MuhI  males  ^tar(  out  in  re* 
tigious  school,  gradual 
Winnowing  process,  suc- 
cessful srudents  often  en- 
ter ciergv  or  community 
elect 

Universal  primary  and  secondary 
ediicaiinn,  many  individuals 
li.ive  specialised  posisefondarv 
education  possibility  01  lile-long 
indivnlwal-initialed  education 

(Gardner,  1983/1993,  p.  339) 

In  Teaching  for  Understanding,  Wiske  (1998)  reports  on  a study  from  1988-1985  by 
Harvard  Graduate  School  of  Education  with  local  schools,  and  although  the  research  was 
K-12,  it  informs  us  as  well.  The  foci  of  the  project,  and  thus  the  book,  was  to  define  what 
is  worth  understanding  for  course  topic  organization  - generative  topics;  to  clarify  what 
students  would  understand  by  the  formulation  of  understanding  goals;  and  to  engage 
learners  in  performances  of  understanding,  requiring  them  to  extend,  synthesize,  and 
apply  what  they  know.  The  Harvard  project  required  "rich"  performances  that  required 
students  to  learn  and  express  themselves  through  the  multiple  intelligences  and  forms  of 
expression,  thus  developing  and  demonstrating  understanding.  Einally,  they  measured 
students'  understanding  by  conducting  ongoing  assessment  of  their  performances  - 
powerful  through  frequency,  resulting  in  the  generation  of  information  to  improve  student 
performance  and  planning.  The  study  was  organized  around  teacher  inquiry  to  help 
teachers  analyze,  design,  enact,  and  assess  practice  with  a focus  on  student 
understanding. 
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In  consideration  of  "generative  topics"  for  those  who  may  not  understand  its  meaning,  it 
is  the  choice  of  curriculum  topics  that  provide  the  opportunity  to  engage  in  inquiry  that  is 
open  ended,  where  students  can  be  generative  rather  than  didactic,  and  where  there  is  a 
very  connected,  correct,  and  direct  result  or  answer.  Generative  learning  engages  students 
and  teachers,  interests  and  excites  them,  and  motivates  and  stimulates  them  to  develop 
understanding;  these  topics  are  controversial  and  connectable  to  other  topics  or 
disciplines. 

When  considering  performances  of  understanding,  it  means  exactly  that  students  will 
demonstrate  or  perform  using  what  they  understand  (e.g.,  authentic  tasks  requiring  them 
to  use  what  they  understand).  Performances  should  be  both  generative  and  challenging. 
Their  three  categories  include  Messing  about,  where  the  inquiry  is  informal  and  not 
bound  or  defined  by  discipline-based  methods  and  concepts.  This  serves  to  draw  students 
in  to  hook  them  or  their  interest,  and  students  can  engage  regardless  of  their  prior  level  of 
understanding,  which  helps  them  connect  the  topic  to  their  own  interests  and  previous 
experiences  and  to  set  the  direction  of  their  learning.  Guided  inquiry  engages  students  in 
using  formal  ideas  or  modes  of  inquiry,  those  central  to  the  goals  or  course,  sometimes 
involving  them  in  the  development  of  skills  in  observation,  recording  of  data,  learning 
new  vocabulary,  and  synthesizing  information  related  to  a particular  question.  Teacher 
guidance  assists  students  in  learning  how  to  apply  the  particular  disciplinary  concepts  or 
methods  so  their  performances  can  grow  more  sophisticated  and  complex  as  do  their 
understandings.  Culminating  performances  (e.g.,  projects,  exhibitions)  engage  students  in 
the  demonstration  of  particularly  well  defined  mastery  of  the  designated  goals  or 
outcomes.  These  are  at  the  higher  level  of  Bloom's  Taxonomy.  Preparing  to  teach  for 
understanding  requires  that  one  understand  "understanding."  The  tables  below  clearly 
illuminate  the  authors'  meaning  and  go  far  to  provide  thoughtful  reflection  for  our 
endeavor. 
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Table  A. 6.9:  Four  Dimensions  of  Understanding  and  Their  Features 


KjiowUilgr 

A.  Irnnifnrvud  itUnitwr  Mieft 
To  what  dcp-cf  do  siudenii' 
pcrfoiTDBiJCM  show  that 
warranted  itieorirs  and 
I oncepis  in  the  domain 
have  iranirormed  .MiidenO 
intuitive  ((tliefs-' 


Metlujdi 


A.  Hialthy  sKepUnsn 

To  wliai  degree  do  viudents 
display  a healthy  skepdeum 
toward  their  own  beliefs 
■and  toward  kiinwledge  from 
such  .sinirecs  as  ihcir 
texiboob,  people's 
opinions,  and  nirisages 
in  the  media? 


fhiTposa 

A.  AuHtrmas  nj  t)u  purpraits  of 
knowUdgr 

To  what  degree  do  snidcnt.s 
set  essential  quejiiioiis, 
purposes,  and  interests  that 
drive  inquirv  in  the  domain? 


Forms 

A.  Masleri  of  perfmvtanit  pmw 
To  what  degree  do  students 
display  mastery  of  the  genres 
of  performances  they  engage 
ill,  such  aswridng 
reporu,  giving  prriiem:itii»ns, 
or  preparing  the  .suige  for 
a ptayr 


b Cohfmt  and  rich 
ctmerfituat  wths 
in  what  degrrr  are  students 
able  to  reason  within  richly 
organized  cniicepmal  webs, 
moving  flexibly  lietween 
details  and  overvierw, 
examples  and 
generalizations? 


B.  Budding  kr\mvledpmlhi 
dofnair) 

To  what  degree  do  students 
ii.se  slralegie.s,  melhods. 
techniques,  and  procedures 
similar  lo  those  used  by 
professionals  in  the  domain 
m build  telialile  knowledge? 


B Vvv  Ilf  knmnlMgf 

Tr>  wlial  degree  do  stiidrnu 
irr.ogui/e  a vaiieq'  til  luwsible 
uses  ol  what  they  learn? 

To  what  degree  do  sttidems 
consider  the  coiiseqirrnres 
of  u.siiig  tlieir  kriuwleitge? 


B.  Efftelrur  itir  of  cyrnhot  sysOoju 
To  wlut  degree  do  siiiileois 
explore  dillerrni  .sywhol 
systtons  to  rcpreseni  thdr 
knowledge  In  eiFectnt  and 
t:reaiive  ways^nt  exarriple 
try  usmg  analogies  and 
metaphors,  colors  anci 
shapes,  or  movetnena,* 


C.  Validating  knmvinige  in  the 
domain 

Are  tnith,  goodness,  and 
beaun  dependiuit  on 
authontaiive  as,sertions. 
or  rather  on  publicly 
agiccd-u|)on  uiteria 
sud)  as  using  systematic 
methrwfs,  providing  rational 
arguments,  weaving 
foherrnt  explanations,  and 
negotiating  meanings 
through  careful  dialogue? 


C.  Ouwryhip  and  autonomy 
To  what  degree  do  .studeuia 
evidence  ownership  and 
die  .iinonomy  to  tear  what 
they  knoyv? 

To  what  degree  liaye  sliidrni* 
developed  a personal 
posiuon  around  what 
they  learn? 


C lAncidfratiim  of 
audurnre  and  rimlrtt 
To  what  degree  do  .slmlrnt*' 
performances  show  an 
awareness  of  dir  audieiirr, 
such  .u  the  audience's 
interests,  needs,  ages, 
ciqicnisc,  oi  cultural 
backgummlx? 

To  what  degree  do  they  show 
arvare tiess  nf  the  context  of 
die  cumiiiuniraiion? 


(Wiske,  1998,  p.  184)  See  Wiske  for  further  breakout. 


What  we  hope  to  accomplish  with  our  professors  is  multi-faceted  but  over  generalized. 
We  hope  to  move  from  teacher  centered  to  learner  centered  courses,  where  professors  are 
facilitators  of  learning  and  there  is  a balance  of  focus  between  learner,  knowledge, 
assessment,  and  teacher  (professor)  (Farquharson,  1995). 
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Figure  A. 6. 25:  The  Teaehing-to-Facilitating  Continuum 


.^1 Source  of  Meaning ► 

(Farquharson,  1996,  p.  66) 

In  an  environment  where  students  are  the  eenter  of  learning  and  professor  facilitators, 
professors,  as  teachers,  need  to  demonstrate: 


• Expertise: 


• Empathy: 


• Enthusiasm: 


• Clarity: 


The  Power  of  Knowledge  and  Preparation 

Connecting  material  to  learner's  life  experience 

Adjusting  content  to  allow  for  spontaneity  and  creativity 

Preparing  teaching  materials  that  build  on  motivation 

The  Power  of  Understanding  and  Consideration 

Checking  out  learner  needs  and  expectations 

Shaping  teaching  to  the  learner's  capacities 

Constantly  considering  the  learner's  perspective 

The  Power  of  commitment  and  Animation 

Having  a personal  investment  in  what  is  taught 

Expressing  emotion,  animation,  and  energy 

The  Power  of  Eanguage  and  Organization 

Ensuring  that  learners  understand  the  logical  connection  of  ideas 

Continuously  checking  learner  comprehension  (Earquharson,  1995, 

p.74  as  cited  in  Wlodkowski,  1991,  pp.  16-43) 


Critical  reflection  involves  assumption  hunting,  as  our  assumptions  are  our  beliefs  that 
give  meaning  and  purpose  to  who  we  are  and  what  we  do  (Brookfield,  1995).  It  is 
important  to  become  aware  of  our  implicit  assumptions  and  how  they  frame  how  we 
think  and  act  - a challenging  intellectual  puzzle.  We  instinctively  resist  challenging  our 
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assumptions  for  fear  of  not  liking  what  we  might  discover  - clarifying  and  questioning 
what  we  have  lived  by  for  a long  time.  Brookfield  identifies  three  categories  of 
assumptions: 

• Paradigmatic  - the  most  difficult  to  uncover,  the  basic  structuring  axioms  used  by 
each  of  us  to  order  the  world  into  what  we  perceive  as  its  fundamental  categories. 
We  may  perceive  these  as  objective,  valid,  and  reality,  facts  we  feel  true.  Often  it 
takes  great  evidence  to  the  contrary  or  a major  disconfirming  experience  to 
achieve  a shift  of  these  assumptions.  But  when  they  are  changed,  there  is  usually 
an  explosive  consequence.  Senge  (1990)  may  call  this  type  of  shift  in  mind, 
metanoia.  Learning  that  causes  us  to  recreate  ourselves. 

• Prescriptive  - what  we  believe  ought  to  happen  in  particular  situations;  these  are 
grounded  in  or  extensions  the  paradigmatic  assumptions. 

• Causal  - explains  how  different  parts  of  the  world  work,  the  conditions  required 
for  changing  its  processes.  Causal  assumptions  are  usually  stated  in  predictive 
terms,  almost  as  "if"  statements.  If  this  is  true,  then  this  will  happen.  For  example, 
"If  we  make  mistakes  in  front  of  students,  this  creates  a trustful  environment  for 
learning  in  which  students  feel  free  to  make  errors  with  no  fear  of  censure  or 
embarrassment.”  Causal  assumptions  are  the  most  easily  uncovered. 

“Discovering  and  investigating  our  causal  assumptions  [are]  the  beginning  of  the 
reflective  process;  the  process  is  not  complete  without  examining  our  prescriptive  and 
ultimately  the  deeper  paradigmatic  assumptions,  those  invisible  rules  that  bind” 
(Brookfield,  1995,  p.  3).  When  it  comes  to  teaching,  we  operate  according  to  our 
assumptions  or  beliefs.  Without  reflection,  we  risk  making  poor  judgments  and  choices. 
Consider  the  following  examples  with  corresponding  comments  after  each  one: 

• Of  course  we  know  what  is  going  on  in  our  classrooms,  after  all,  we've  been 
doing  this  for  a number  of  years,  haven't  we?  This  unexamined  common  sense 
assumption  is  a notoriously  unreliable  guide  to  action,  (p.  4) 

Critically  reflective  teachers  recognize  that  good  teaching  is  not  always  evaluated  as  good 
by  students.  The  diversity  of  students  in  today's  classrooms  makes  it  impossible  have 
uniform  results.  Also  good  performance  evaluations  of  perfect  scores  may  not  serve 
students’  interests;  it  could  imply  that  we  are  not  challenging  students.  Someone  who 
scores  a perfect  "10"  may  be  presumed  (or  is  as  likely)  to  be  doing  something  wrong  as 
right.  The  dynamics  and  contradictions  of  teaching  cannot  be  reduced  to  quantification. 
And  if  we  want  to  change  our  teaching,  we  must  realize  that  the  answers  are  not  always 
"out  there"  available  in  some  workshop,  book,  or  article.  Instead  we  must  research  our 
own  teaching  and  practices,  imported  ones  rarely  fit  faultlessly.  There  are  not  any  simple 
answers;  therefore,  it  is  important  to  note  that 

significant  learning  and  critical  thinking  inevitably  induce  an  ambivalent  mix  of 
feelings  and  emotions,  in  which  anger  and  confusion  are  as  prominent  as  pleasure 
and  clarity. . . [The  rule]  that  the  customer  is  always  right  - is  often  pedagogically 
wrong. . . .Equating  good  teaching  with  a widespread  feeling  among  students  that 
you  have  done  what  they  wanted  ignores  the  dynamics  of  teaching  and  prevents 
significant  learning.  (Brookfield,  1995,  p.  21) 
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So,  why  is  critical  reflection  so  important: 

it  helps  us  to  take  informed  actions;  to  develop  a rationale  for  practice;  to  avoid 
self-laceration;  it  grounds  us  emotionally,  enlivens  our  classrooms,  increases 
demoeratie  trust;  and,  although  ideologieal,  eritieal  refleetion  is  morally  grounded 
as  it  is  the  basis  for  the  ereation  of  eonditions  within  whieh  people  ean 
learn. . .anchored  in  values  of  justice,  fairness,  and  compassion,  critical  reflection 
finds  it  political  representation  in  the  democratic  process. . .In  pedagogic  terms, 
this  means  the  ereation  of  demoeratie  elassrooms. . .In  terms  of  professional 
development,  it  means  an  engagement  in  eritieal  eonversation.  (pp.  26-27) 


Figure  A. 6. 26:  The  Critieal  Refleetion  Proeess 


To  assist  in  eritieal  refleetion,  the  following  questions  ean  be  helpful.  See  The  Classroom 
Critical  Incident  Questionnaire  below. 
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Figure  A. 6. 27:  The  Classroom  Critical  Incident  Questionnaire 


The  Classroom  Critical  Incident  Questionnaire 

Please  take  about  five  minutes  to  respond  to  each  of  the  questions  below  about 
this  week’s  class(es).  Don’t  put  your  name  on  the  form — ^your  responses  are 
anonymous.  When  you  have  finished  writing,  put  one  copy  of  the  form  on 
the  table  by  the  door  and  keep  the  other  copy  for  yourself.  At  the  start  of  next 
week’s  class,  I will  be  sharing  the  responses  with  the  group.  Thanks  for  taking 
the  time  to  do  this.  What  you  write  will  help  me  make  the  class  more  respon- 
sive to  your  concerns. 

1.  At  what  moment  in  the  class  this  week  did  you  feel  most  engaged  with 
what  was  happening? 

2.  At  what  moment  in  the  class  this  week  did  you  feel  most  distanced  frorti 
what  was  happening? 

3.  What  action  that  anyone  (teacher  or  student)  took  in  class  this  week  did 
you  find  most  affirming  and  helpful? 

4.  What  action  that  anyone  (teacher  or  student)  took  in  class  this  week  did 
you  find  most  puzzling  or  confusing? 

5.  What  about  the  class  this  week  surprised  you  the  most?  (This  could  be 
something  about  your  own  reactions  to  what  went  on,  or  something 
that  someone  did,  or  anything  else  that  occurs  to  you.) 

(Brookfield,  1995,  p.  115) 

Brookfield's  (1995)  model  for  critical  reflection  uses  four  critically  reflective  lenses 
through  which  to  view  teaching: 

• Autobiographies  as  Learners  and  Teachers  - This  puts  us  in  the  role  of  'other.'  We 
see  our  practice  from  the  other  side  of  the  mirror,  and  we  become  viscerally 
connected  to  what  our  own  students  are  experiencing  - the  first  step  on  the  critical 
path.  Through  personal  self-reflection,  we  become  aware  of  the  paradigmatic 
assumptions  and  instinctive  reasoning  within  which  we  work.  Once  we  realize 
what  these  are,  it  is  possible  to  test  them  for  their  accuracy  and  validity  through 
conversations  with  others. 

• Students'  Eyes  - Experiencing  ourselves  as  our  students  see  us  makes  us  more 
aware  of  our  actions  and  the  underlying  assumptions;  this  can  serve  to  confirm  or 
challenge  the  existing  power  relationships  in  the  classroom  and  whether  students 
are  experiencing  the  "meanings"  that  we  intend  from  our  practice. 

• Colleagues'  Experiences  - Peer  observation  can  go  far  to  see  one's  self  through  the 
eyes  of  colleagues,  especially  if  we  engage  in  critical  conversations,  as  then  we 
can  consider  aspects  of  our  practice  that  are  revealed  through  that  observation  and 
the  conversations;  often  this  process  will  reveal  things  that  we  ourselves  cannot 
see  in  or  about  ourselves.  When  examining  what  they  observed,  we  can  view  our 
practice  through  yet  another  lens. 
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• Theoretical  Literature  - Exploring  the  theoretical  literature  helps  us  to  interpret 
what  we  are  doing  in  multiple  ways,  e.g.  naming  what  we  are  doing  in  different 
ways;  illuminating  generic  aspects  of  what  we  though  idiosyncratic  events  and 
processes.  It  can  confirm  what  we  are  and  should  be  doing,  or  it  can  contradict  us 
and  reveal  other  methods  and  processes  to  consider,  (pp.  29-39) 

When  engaging  in  the  processes  described  above,  considering  our  practice  from  the 
lenses  of  others  and  the  literature  can  be  emotional,  sometimes  even  painful.  Critical 
reflection,  like  that  of  our  students  when  learning  new  knowledge,  involves  the 
negotiation  of  our  feelings  because  we  are  examining  our  practice  critically.  "Our 
practice  becomes  the  object  of  systematic  inquiry"  (p.  39).  Throughout  the  inquiry,  we 
realize  that  there  may  be  issues  of  power  and  control  in  our  classrooms  that  prevent  us 
from  engaging  students  in  ways  that  are  more  meaningful.  To  "empower"  students 
requires  actually  "giving  up"  our  own  power.  In  my  opinion,  power  is  finite;  if  we 
authorize,  or  empower,  students  to  take  responsibility  for  their  own  learning  and  growth 
while  we  provide  structured  opportunities  and  facilitate  the  learning  processes,  then  we 
must  truly  empower  them  in  a way  where  they  can  make  decisions  on  their  own  or  in 
cooperative  groups.  We  must  honor  those  decisions,  give  up  our  power  (and  its 
corresponding  authority)  of  making  decisions  for  them.  This  means  that  we  permit  them 
to  make  mistakes;  but  remember,  great  learning  can  come  out  of  academic  mistakes.  The 
problem  or  project  is  not  the  end  in  itself;  it  is  the  process  through  which  to  learn.  So  as 
we  learn  to  view  ourselves  and  our  practice  through  a variety  of  lenses,  our  own  lens  will 
become  more  honest  and  revealing,  more  objective,  and  we  will  be  more  able  to  see  our 
practice  for  what  it  is  and,  more  importantly,  what  it  can  become.  In  terms  from 
Brookfield  (1995),  we  become  more  able  to  admit  to  the  more  oppressive  aspects  of  our 
actions  (in  teaching  practice)  and  move  toward  the  creation  of  a more  democratic 
classroom  in  which  students  have  real  power.  We  are  more  able  to  confront  the  issues  and 
dilemmas,  the  contradictions  that  engage  us  daily  and  become  better  at  realizing  rationale 
to  justify  our  methods  and  practice.  We  move  from  self  ignorance,  in  my  own 
terminology,  to  realizing  that  teaching  is  ideological  in  nature,  that  curricula  is 
determined  by  individuals  and  should  be  perceived  as  constructed  and  tentative,  and  that 
it  can  be  dismantled  and  reframed  (and  should  be  in  my  opinion)  quite  often. 

It  horrifies  me  that  I know  professors  who  rarely  if  ever  change  their  course  curricula, 
unless  technology  or  scientific  advances  force  them  to.  And,  even  then,  I know  of  some 
who  teach  what  they  "frame"  as  the  basics  and  do  not  make  those  changes.  Of  course,  this 
is  the  negative  side  of  the  coin,  but  if  we  are  honest  with  ourselves,  we  all  realize  there  is 
so  much  more  we  can  do  with  the  curricula  content!  For  example,  how  often  do 
professors  actually  engage  in  the  validation  of  their  courses  and  entire  curricula?  Most 
are  driven  by  commercial  textbooks  that  are  rarely  good  or  well  developed.  When  that  is 
the  case,  I am  not  sure  why  we  need  professors.  Our  students  are  entirely  capable  of 
reading  the  text  and  learning  what  it  has  to  offer  with  some  assignments.  When 
considering  curricula,  they  are  chosen  by  individuals  or  groups  with  their  own  agendas 
and  interests.  What  they  entail  as  content  comes  out  of  those  agendas  and  interests,  or 
lack  of,  and  thus  the  reliance  on  the  textbooks. 
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Teaching  is  ideological  if  someone  truly  appreciates  the  arts  and  practice  of  teaching. 
Because  of  power  structures,  moving  towards  democratic  processes  to  determine 
curricula  may  threaten  some  who  are  in  a "comfort  zone"  or  "familiar  routines"  (p.  41). 
There  is  a risk  in  suggesting  change  in  curricula  or  teaching  processes.  Colleagues  may 
take  their  assumptions  for  granted,  but  if  we  are  to  survive  and  thrive,  we  must  as 
unthreateningly  as  possible  challenge  others  to  question  their  own  assumptions  - as  we 
are  now  doing.  We  must  learn  to  minimize  risk  by  being  tactically  astute  and  cunning 
when  moving  toward  getting  others  to  challenge  common  assumptions  so  they  in  turn 
will  make  changes  in  their  teaching  practice:  challenge  as  non-threateningly  as  possible, 
as  non-confrontationally  as  possible,  yet  in  a way  that  moves  the  challenges  forward  to 
bear  fruit  and  change  where  needed.  This  might  be  known  as  political  astuteness. 

At  NIU,  we  have  a great  deal  of  academic  and  personal  freedom  with  teaching 
methodology  and  our  curricula.  We  do  not  have  as  a great  a risk  in  challenging  the  status 
quo  as  in  some  other  institutions.  There  is  no  risk  in  the  changes  that  we  are  seeking 
through  this  initiative,  and  there  is  a great  deal  of  administrative  support.  However,  it  is 
important  to  note  the  above  statements  as  there  are  many  individuals  who  assume  their 
teaching  practice  is  worthy,  when  in  fact  it  is  not,  so  we  are  dealing  more  with  the 
challenge  of  motivating  ineffective  teaching  professors  to  change  to  more  effective 
practice  than  anything  else. 

When  engaging  in  critical  reflection  as  a teaching  professional,  our  thinking  becomes 
different;  we  begin  to  see  value  in  professional  development  and  feel  a need  to  continue 
evolving.  We  begin  to  realize  that  we  will  never  be  "there"  and  that  our  ideas  and 
practices  will  always  need  investigation.  Our  practice  is  an  ongoing  investigation. 
Therefore,  we  will  enter  into  a new  life  cycle,  one  where  we  assess,  take  action,  consider 
the  results  of  that  action,  and  then  again  engage  in  investigation.  The  following  model  is 
a good  one,  whether  for  ourselves  or  with  our  students. 

Human  problem  solving  involves  beginning  with  prior  outcomes  or  consequences  of 
prior  actions.  These  constitute  the  current  problems  or  opportunities.  When  engaging  in 
solving  the  problems  or  exploiting  the  opportunities,  we  use  resources  and  technological 
processes,  resulting,  yet  again,  in  another  set  of  outcomes  or  consequences,  which  then 
present  themselves  as  new  problems  or  opportunities.  Although  this  model  is  presented  as 
a conceptual  framework  for  technology  education,  to  me  it  is  a simple  and  somewhat 
generic  model  that  works  as  a conceptual  framework  for  solving  problems,  regardless  of 
where  used  (Savage  & Sterry,  1990).  This  model  also  works  in  the  context  of  "problem- 
based  learning  and  problem  solving"  discussed  above. 
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Figure  A. 6. 28:  A Model  for  Technology  Education 


Therefore,  we,  and  our  teaching,  are  in  continual  formation  (evolving);  our  teaching 
becomes  a connective  activity,  meaning  that  our  approach  has  now  become  one  of 
inquiry,  always  needing  investigation  and  constantly  forming  or  evolving.  We  learn, 
modify,  and  continue  to  change  our  practice,  sometimes  with  contrary  rhythms  to  those 
before  - meaning  one  rhythm  might  develop  as  we  study  our  own  teaching  and  another 
may  develop  as  we  study  how  our  students  experience  our  practice.  We  make  changes 
that  enhance  our  connectivity  to  students  as  learners.  We  connect  educational  processes 
and  students’  experiences  of  learning  and  what  students  feel  becomes  a guiding  principle. 
Hopefully,  we  arrive  at  a vision  of  democratic  education,  where  the  power  has  shifted  and 
is  shared.  We  achieve  critical  responsiveness  that  is  driven  by  clearly  articulated  values 
and  that  is  creative  to  the  needs  and  concerns  of  students.  We  engage  in  fluctuating 
rhythms  as  we  respond,  making  sure  that  what  we  do  is  grounded  in  an  accurate 
understanding  of  what  students  are  experiencing  or  not  experiencing.  To  ensure  that  we 
are  accurate,  we  engage  with  students  in  their  critical  reflections  about  what  they  are 
experiencing,  how  they  connect  or  how  they  do  not,  why  they  disengage,  and  explore 
why  sometimes  our  actions  puzzle  them  or  affirm  them.  This  involves  us  in  continuous 
exploration  of  connections,  why  they  happen  or  do  not,  and  goes  far  to  explain  the  effects 
of  our  practice.  In  doing  this,  we  begin  to  realize  our  own  voices  and  those  of  our 
students.  Democratic  classrooms  provide  the  opportunity  for  those  voices  to  emerge.  We 
begin  to  speak  about  our  practice,  authenticating  it.  As  we  speak  about  it,  we  learn  about 
it  even  more  deeply  and  become  capable  of  identifying  irrelevant  "other"  voices  that  are 
external  with  external  interests.  We  begin  to  listen  and  hear  our  own  voices  and  those  of 
our  students,  using  this  to  guide  our  practice.  We  begin  not  to  seek  that  external  approval 
of  authority  but  rather  what  works.  Mezirow  (1991)  calls  "those  deeply  embedded 
internal  injunctions  that  define  the  boundaries  of  what  we  allow  ourselves  to  think  - 
premise  distortions.  They  are  self-censorship  devices  - nagging  voices  of  denial  that  set 
out  acceptable  interpretations  of  classroom  events  and  that  remind  us  constantly  of  what 
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practices  are  off-limits"  (as  cited  in  Brookfield,  1995,  p.  45).  As  we  mature  in  our  ability 
to  reflect  critically,  we  begin  to  recognize  these  "jarring  and  dissonant"  voices  as  those 
with  false  messages  that  are  refuted  by  our  practice.  They  simply  bind  us  into  ineffective 
practice. 

As  important  as  recognizing  those  limiting  voices  is  hearing  those  voices  who  speak  in 
alternate  ways,  those  other  voices  who  help  us  to  take  our  own  inner  voices  seriously. 

For  example,  when  someone  speaks  and  you  realize  that  they  are  saying  what  you  have 
been  struggling  to  articulate,  ultimately  affirming  something  you  believe  in  but  have  not 
put  into  words.  These  voices  are  important  because  they  help  you  to  admit  out  loud 
something  you  have  hesitated  to  admit.  The  discovery,  honoring,  and  expression  of 
authentic  voice  is  transforming.  When  we  speak  authentically  or  hear  an  authentic  voice, 
our  experience  and  our  practice  coheres  in  a way  that  is  consistent.  Finding  our  voice, 
about  our  practice,  is  powerful  and  fundamentally  connected  with  developing  our  sense 
of  agency.  Once  we  can  talk  about  and  name  our  experiences  in  teaching,  we  begin  to 
understand  what  we  believe,  what  we  do  not,  and  why.  This  empowers  us,  gives  us  a 
source  of  action  generated  from  within  rather  than  imposed  from  external  sources.  We  act 
with  intent,  drawing  from  our  own  knowing.  We  are  critical,  and  we  create  our  world,  no 
longer  merely  responding  or  reacting  to  it.  Agency  means  the  connection  between 
reflection  and  action.  Power  comes  from  intentional  action,  and  we  act  on  what  we  know 
and  believe.  Hopefully,  those  of  us  engaged  in  this  initiative  are  discovering  or  deepening 
our  understanding  of  our  voices  about  teaching  practice,  and  through  the  processes  of 
analysis,  learning,  development,  and  intentional  changes  and  decision  making,  we  are 
becoming  more  powerful  in  our  practice  and  our  students  will  become  more  powerful  as 
we  move  into  more  democratic  practices  and  sharing  of  power. 
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12.  Student  Performance  Assessment* 

Jule  Dee  Scarborough 

Foundation 

The  Education  Commission  of  the  States  (1995)  has  identified  the  attributes  of  good  practice  for 
higher  education,  but  they  apply  equally  to  secondary  education.  The  commission  argues  that 
good  assessment  can  lead  to  the  transformation  of  education  when  considering  . .that  when 
[educators]  systematically  engage  in  these  good  practices,  student  performance  and  satisfaction 
will  improve”  (p.5).  These  quality  indicators  show  the  integral  aspect  of  assessment.  They 
provide  evidence  of  successful  teaching  and  learning  or  evidence  for  setting  improvement  goals 
and  are  also  helpful  in  guiding  and  structuring  improvements  (Huba  & Freed,  2000): 

Quality  begins  with  an  organization  culture  that  values 

1 . High  expectations 

2.  Respect  for  diverse  talents  and  learning  styles 

3.  Emphasis  on  the  early  years  of  study  [e.g.,  general  education] 

A quality  curriculum  requires 

4.  Coherence  in  learning 

5.  Synthesizing  experiences 

6.  Ongoing  practice  of  learned  skills 

7.  Integrating  education  and  experience 

Quality  instruction  builds  in 

8.  Active  learning 

9.  Assessment  and  prompt  feedback 

10.  Collaboration 

1 1 . Adequate  time  on  task 

12.  Out-of-class  contact  with  faculty  (possibly  an  exception  for  secondary  school 
teachers) 

Wiggins  and  McTighe  (2000)  provide  a “backwards”  design  process  where  student 
learning  standards  are  considered  as  the  foundation  for  all  curriculum  and  instruction  decisions 
and  are  followed  by  choosing  the  assessment  procedure  that  will  provide  the  desired  type  of 
evidence  of  learning.  Wiggins  (1998)  identifies  three  types  of  educational  standards:  “Content 
standards:  What  should  students  know  and  be  able  to  do?  Performance  standards:  How  well 
must  students  do  their  work?  And  Task  (work- design)  standards:  What  is  worthy  and  rigorous 


Conard  White  served  as  primary  leader  of  this  program  component. 
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work?  What  tasks  should  students  be  able  to  do?”  (p.  106).  Wiggins  and  McTighe  state  (pp. 
110-111)  that  assessments  should  be 

1.  Credible  to  all  stakeholders,  but  especially  to  teachers,  parents,  and  older  students. 

2.  Useful,  meaning  user-friendly  and  helpful  to  the  student  performers  and  their 
coaches,  namely  teachers. 

3.  Balanced,  in  the  use  of  all  assessment  methods,  to  provide  a rich,  defensible  and 
feasible  profile  of  achievement,  but  anchored  in  authentic  and  complex  performance 
tasks. 

4.  Honest  yet  fair,  reporting  how  each  student  is  doing  against  important  standards  but 
not  uselessly  ranking  novices  and  experts  against  each  other. 

5.  Intellectually  rigorous  and  thought  provoking;  focusing  on  core  ideas,  questions, 
problems,  texts,  and  knowledge;  but  also  engaging  and  stimulating  of  inquiry  and 
interest  in  intellectual  work. 

6.  Feasible  in  terms  of  resources,  logistics,  politics,  and  redeployment  of  time  for 
collaboratively  designing,  debugging,  using,  evaluating,  and  reporting  work. 

Lissitz  and  Schafer  (2002,  pp.  23-26)  also  provide  standards  for  assessment  quality: 

1 . Quality  assessments  arise  from  and  accurately  reflect  clearly  specified  and 
appropriate  achievement  expectations  for  students. 

2.  Sound  assessments  are  specifically  designed  to  serve  instructional  purposes. 

3.  Quality  assessments  accurately  reflect  the  intended  target  and  serve  the 
intended  purpose. 

4.  Quality  assessments  provide  a representative  sample  of  student  performance  that  is 
sufficient  in  its  scope  to  permit  confident  conclusions  about  student  achievement. 

5.  Sound  assessments  are  designed,  developed,  and  used  in  such  a manner  as  to 
eliminate  sources  of  bias  or  distortion  that  interfere  with  the  accuracy  of  results. 

Kuhs  et  al.  (2001,  p.  2)  add  that  “in  addition  to  guiding  classroom  instruction,”  assessment  helps 
teachers 

• formulate  plans  and  strategies  to  support  the  instructional  needs  of  students 

• share  information  with  students  about  their  progress 

• collect  information  to  assign  student  grades 

• evaluate  the  effectiveness  of  their  instructional  strategies  and  curricula 

• prepare  summative  information  on  student  progress  for  decisions  such  as 
promotion,  retention,  assignment  to  special  programs,  and  referrals  to  other 
needed  assistance  programs. 

Additionally,  they  (p.  4)  offer  characteristics  of  quality  assessment  through  the  following 
questions: 
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1 . Does  the  assessment  focus  on  knowledge  and  skills  that  were  taught  in  class  and  are 
outlined  in  district  curriculum  guides  and  in  state  and  national  content  standards? 

2.  Does  the  assessment  provide  information  about  student  learning  that  represents 
typical  performance? 

3.  Does  the  assessment  provide  opportunities  for  all  types  of  students  to  demonstrate 
what  they  have  learned? 

These  authors  also  discuss  the  types  of  assessments,  observation,  performance  tasks, 
scoring- guides  checklists  and  rubrics,  tests,  portfolios,  and  interviews,  as  well  as  what  they  term 
a multifaceted  assessment  system  (Figure  1). 


Figure  12.1  Multifaceted  Assessment  System  (Kuhs  et  al.,  2001,  p.  157) 

Teachers  rely  greatly  upon  “teacher- made  tests.”  Most  of  the  teachers  we  worked  with  identified 
test  construction  as  an  area  in  which  they  felt  little  confidence  about  their  own  capabilities.  The 
second  area  within  testing  is  grading.  The  meaning  of  grades  within  a single  course  by  one 
teacher,  and  across  several  courses  by  several  teachers,  as  judgments  about  student  learning  or 
achievement  can  be  problematic  when  considering  all  the  factors  and  issues.  And  finally, 
teachers  are  aware  of  a current  focus  on  student  performance  assessment  versus  traditional 
testing,  but  not  of  how  the  two  relate;  therefore,  how  to  develop  performance  tests  and  rubrics 
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has  also  been  an  important  area  for  professional  development.  Teachers  are  somewhat  confused 
by  the  movement  toward  performance  assessment,  while  the  continued  reliance  of  schools  on 
standardized  testing  makes  them  question  the  value  of  active  learning  and  performance-based 
assessment.  Most  teachers  understand  the  test  as  the  terminal  measurement  of  student 
achievement  rather  than  understanding  that  tests  or  “test  items  are  meant  to  be  useful  as 
indicators  of  valued  ‘real  world’  performances”  (Linn  & Baker,  1996,  as  quoted  in  National 
Society  for  the  Study  of  Education,  1996,  p.  85).  Therefore,  performance  assessments  should 
create  additional  opportunity  for  students  to  provide  evidence  they  can  use  the  knowledge 
measured  on  the  test. 

Wiggins  (1998)  defines  “educative  assessment”  as  “deliberately  designed  to  teach  (not 
just  measure)  by  revealing  to  students  what  worthy  adult  work  looks  like  (offering  them 
authentic  tasks).  It  should  provide  rich  and  useful  feedback  to  all  students  and  to  their  teachers, 
and  it  should  be  designed  to  assess  the  use  of  feedback  by  both  students  and  teachers”  (p.  12).  A 
learning- centered  assessment  system  should 

1 . Be  designed  to  improve  performance  (of  student  and  teacher) . . .built  upon  a bedrock  of 

meaningful  performance  tasks  that  are  credible  and  realistic  (authentic),  hence  engaging 

students.  This  system  must  also: 

a.  Be  open  - that  is  based  on  tasks,  criteria,  and  standards  known  to  students  and  their 
teachers... less  reliant  on  audit  testing  methods  that  require  test  questions  are  kept  a 
secret. 

b.  Model  exemplary  instruction,  encouraging  rather  than  undercutting  desirable  teaching 
practice  and  showing  all  teachers  how  the  most  rich  and  vital  educational  goals  and 
pedagogies  can  be  assessed. 

c.  Use  grades  that  stand  for  something  clear,  stable,  and  valid. . .linked  directly  to 
credible  and  important  state  or  national  standards  for  assessing  performance  on 
authentic  tasks. 

d.  Measurably  improve  student  performance  over  time  so  that  standards  once  thought 
very  high  and  reachable  by  only  a few  become  reasonable  expectations  for  many 
students. 

2.  Provide  useful  feedback  to  students,  teachers,  administrators,  and  policymakers.  A useful 

feedback  system  must: 

a.  Provide  data  and  commentary  that  are  rich,  clear,  and  direct  enough  to  students  and 
teachers  to  self- assess  accurately  and  self- correct  their  own  performances 
increasingly  over  time. . . .not  center  on  praise  and  blame. 

b.  Provide  ample  opportunities  to  get  and  use  timely  and  ongoing  feedback.  (Wiggins, 
1998,  pp.  12-13) 

Wiggins  (pp.  17-18)  presents  five  key  ideas  on  assessment  and  its  reform: 

1.  Assessment  must  center  on  purpose,  rather  than  merely  on  techniques  or  tools. 

2.  Assessment  is  a moral  matter  in  that  teachers  and  students  are  entitled  to  systems  that 

are  user  friendly  and  enhance  teaching. 
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3.  Assessment  is  central  to  instruction,  not  peripheral  to  it. 

4.  Assessment  “anchors  teaching,  and  authentic  tasks  anchor  assessment.” 

5.  Performance  improvement  is  local.  Although  guided  by  national  and  state  standards, 

student  learning  occurs  locally. 

A Balanced  Assessment  System 

The  best  approach  is  to  use  a variety  of  assessment  tools  or  procedures  to  produce  a balanced 
system  over  the  course  of  the  standards  to  be  achieved.  The  idea  is  to  offset  the  limitations  of 
one  type  of  assessment  against  the  strengths  of  another. 

A balanced  system  should  include  good  and  reliable  traditional  assessment,  performance 
assessment,  and  portfolio  assessment.  Traditional  assessment  is  typically  used  to  determine 
grades  and  rankings,  whereas  performance  assessment  provides  the  opportunity  to  observe 
learning  results,  and  portfolio  assessment  provides  the  opportunity  to  determine  growth  and 
development  over  time.  A system  encompassing  various  assessments  can  develop  students’ 
ability  to  perform  in  types  of  assessment  where  they  have  traditionally  shown  weaker 
performance,  while  also  giving  them  the  opportunity  to  perform  through  procedures  in  which 
they  have  excelled  (Wiggins,  1998,  pp.  1 15- 1 16).  (See  Figures  2 and  3.) 
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Figure  12.2  Typical  Unbalanced  Assessment  in  Classrooms  and  Schools  (Wiggins,  1998, 
p.  115) 
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Figure  12.3  Exemplary  Assessment  Balance  (Wiggins,  1998,  p.  116) 

When  providing  professional  development  for  teachers,  it  is  impossible  to  address  all 
student  assessment  issues  in  a single  program.  Therefore,  we  chose  to  focus  on  improving  their 
traditional  test  development  skills  by  working  with  them  on  the  development  of  their  module 
pretests  and  posttests,  making  sure  that  those  tests  were  measuring  student  progress  toward 
achieving  standards.  However,  to  introduce  them  formally  to  more  authentic  performance 
assessment,  we  helped  them  understand  what  authentic  means,  what  a performance  is,  and  how 
to  identify  performances  that  provide  evidence  of  student  learning  of  vital  concepts  and 
principles.  This  is  where  providing  experiences  in  real-world  industry,  business,  or  community 
organizations  helped  them  transform  “theory  into  action.” 

Many  teachers  struggle  to  distinguish  among  (1)  knowledge,  the  concepts  or  principles  of 
a discipline;  (2)  cognitive  skills  that  transcend  all  disciplines  and  learning;  and  (3)  other  types  of 
skills,  ranging  from  academic  skills  (such  as  reading),  the  problem-solving  process,  the 
organization  of  information,  and  beyond  that,  discipline- specific  skills.  Of  course,  there  is  a fine 
line  between  labeling  some  of  these  “knowledge”  versus  “skills.”  In  addition  and  very 
importantly,  many  teachers  are  not  secure  in  knowing  the  difference  between  the  knowledge 
taxonomy  of  their  discipline  and  the  more  contextual  “curriculum  content”  within  which 
knowledge  or  skills  can  be  embedded  or  taught.  For  example,  teachers  may  identify  computer- 
aided- design  (CAD)  software  functions  as  knowledge,  whereas  mechanical  drawing  or 
engineering  graphics  concepts  such  as  cross-sectional  views  would  be  a concept,  and  the 
software  functions  and  hardware  components  of  CAD  would  be  technological  tools.  Another 
example  is  from  English:  a knowledge  concept  would  be  figurative  or  persuasive  writing, 
whereas  the  novel  would  be  the  contextual  content  within  which  the  concepts  and  principles  of 
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writing  are  applied.  We  have  often  found  teachers  testing  on  the  novel  rather  than  the 
knowledge,  concepts,  and  principles  involved  in  writing  literature. 

Education  versus  training  is  something  else  to  consider.  The  intent  of  educators  is  to 
teach  knowledge  that  can  be  used  across  learning  (courses  or  disciplines)  or  real-world  contexts. 
Use  of  a concept  or  principle  requiring  interpretation,  reasoning,  practical  application,  and 
comprehension  at  the  Bloom’s  Taxonomy  levels  of  synthesis  and  evaluation,  or  the  “create” 
level  of  the  revised  Bloom’s  Taxonomy,  takes  place  through  education  processes  (Anderson  & 
Krathwohl,  2001).  Education  is  often  combined  with  training.  Eor  example,  education  should 
occur  on  mechanical  drawing  knowledge,  while  training  should  occur  on  the  CAD  software  and 
hardware.  An  example  of  training  without  education  is  when  students  use  a “template”  approach 
to  algebraic  formulae  but  are  unable  to  find  a mistake  because  they  have  no  understanding  of  the 
relationships  between  the  factors  in  the  formula. 

The  standards  movement  has  made  it  easier  to  bring  teachers  to  an  understanding  about 
what  they  should  be  measuring.  When  teachers  understand  the  critical  questions  to  ask 
themselves  - what  standards  they  want  students  to  achieve,  what  level  of  achievement  is  desired, 
what  the  best  method  of  assessing  the  learning  is,  and  ultimately  how  to  contextualize  what  is  to 
be  learned  - they  can  then  move  on  to  other  instructional  decision  making.  They  begin  to 
understand  that  performance  assessment  should  be  intertwined  with  learning.  Traditional  testing 
is  artificial  and  requires  a “time  out”  for  testing  memory  or  comprehension,  such  as  making 
simple  comparisons.  Many  teachers  are  striving  to  infuse  more  critical  thinking  into  traditional 
test  items;  others  are  rewriting  traditional  test  items  to  ensure  concept  attainment  when  allowing 
use  of  technology  while  taking  the  test.  Authentic  and  performance-based  assessment,  however, 
differs  in  that  often  students  will  not  feel  that  they  are  being  tested  but  that  they  are  performing 
while  learning  and  learning  while  performing.  The  two  are  not  artificially  separated.  It  is  more  a 
culture  of  achieving  outcomes  through  performance  tasks.  To  us,  a complex  performance  task  is 
one  that  embeds  several  performances  as  a cluster;  that  cluster  is  embedded  within  a real-world 
scenario  and  the  performance  tasks  are  as  authentic  as  possible.  The  performances  require  a 
range  of  outcomes,  such  as  the  use  of  knowledge  at  the  upper  critical  thinking  levels  and  the  use 
of  cognitive  skills,  and  may  include  a requirement  that  students  use  academic  and  other  skills 
within  a context  to  which  they  can  relate.  The  question  of  “what”  is  to  be  measured  and  then  the 
alignment  of  the  achievement  targets  with  assessment  methods  is  critical.  Chatterji  (2003,  pp.  96- 
98)  identifies  five  types  of  assessments: 

la.  Written  structured-response  assessments  - usually  timed,  fixed,  or  selected 

response,  written  exercises 

lb. Written  open-ended  assessments  - usually  timed,  constructed  response,  written 
exercises 

2.  Behavior-based  assessments  - behaviors  or  demonstrations  exhibited  in  natural  or 
structured  settings 

3.  Product-based  assessments  - products,  reports,  or  items  created  in  structured  or 
unstructured  situations 

4.  Interview-based  assessments  - one-on-one  verbal  (oral)  interaction  in  structured  or 
unstructured  situations 
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5.  Portfolios  - purposeful  collections  of  behaviors  or  work  samples  made  over  time 


Lissitz  and  Shafer  (2002)  provide  an  example  of  matching  targets  to  methods  (Table  1). 


Table  12.1  Aligning  Achievement  Targets  to  Assessment  Methods  (Lissitz  and  Shafer, 
2002) 
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Wiggins  (1998,  p.  23)  presents  the  key  differences  between  typical  tests  and  authentic 
tasks  (Table  2).  Also  important  is  the  question,  “To  what  level  of  achievement?”  Along  with  the 
original  Bloom’s  Taxonomy  (Table  3)  (Bloom,  1956),  I use  the  revised  Bloom’s  Taxonomy  in  A 
Taxonomy  for  Learning,  Teaching,  and  Assessing  (Anderson  & Krathwohl,  2001,  p.  100).  This 
schema  (Table  4)  identifies  the  types  of  knowledge  and  reveals  changes  in  the  cognitive  process 
dimension.  When  using  this  taxonomy  to  plan  my  own  teaching  and  student  learning,  I may  still 
need  the  “synthesis”  level  of  the  original  Bloom’s  Taxonomy,  but  “create”  as  the  final  category 
is  very  appropriate  and  does  inherently  require  synthesis.  Another  very  useful  aspect  of  A 
Taxonomy  for  Learning,  Teaching,  and  Assessing  is  a table  comparing  the  original  Bloom’s 
Taxonomy  to  other  multidimensional  classification  systems. 
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Table  12.2  Key  Differences  between  Typical  Tests  and  Authentic  Tasks  (Wiggins,  1998, 
p.23) 
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answers,  nr  sldlls 

Are  uiteyrated  challerimrs.  in 
w luch  knowledge  anil  jodymcnl 
must  be  iimovativcly  uso-l  to 
fashion  .1  quatiiv  produitor 
performance. 

The  l.tsk  is  liiultiJaci-teO  .lild  iujii- 
ixnitine.  even  if  Iherv*  i>  a ''riyliT’ 
answet.  It  tlius  requiros  pnililem 
clarilicaiion.  Inal  and  erion  ailiusl- 
mt'ols,  adiiplii'p  a>  the  i asx'  or  lacis 
at  hand,  etc 

Are  MTnplih't>d  so 
as  to  be  easy 
to  M'ort’  rt'hnbly 

Involve  ciMiiplexand  non 
arbitrary  tasks,  cillerij.  .tnd 
standiirds. 

The  task  involves  the  imporknii 
dspei  ts  ul  periojmance  and  - oi 
core  challenges  of  tlio  fieUi  iit  shidy, 
not  the  easily  -.cori'd,  Ji)e>  not 
sarril'icc  validity  for  rch.ibiHty 

Arc  one  shot 

Art.'  Itcr.itive-  uinlain  rccurriiiy, 
cssenti.il  tasks,  ponres.  and 
standards 

fill'  work  is  designed  to  reve.il 
whether  fh«'  student  has  achievvU 
real  versus  ^ssetido  ma.sten’.  or 
imi'lerslanding  versus  mere 
lamiharity,  ox'cr  time. 

Depend  on  hiphlv 
technical  ttjrrelatioiis 

T^roviile  din'd  t-viJcnce, 
involving  tasks  that  have  been 
validated  apainst  cure  ailult 
Miles  and  discipline-bascvi 
chdllenpet. 

Ibe  la.sk  is  valid  and  fair  or  its 
face  It  thus  pi'iiki’s  student  inlnest 
and  persistence,  and  seems  apt  and 
1 hallenpine,  to  *.:iidents  I'm! 

U'iU  tiers 

Pnivide  a score 

Provide  usable,  djapncwlK 
IsOtnetiiiies  toncuitviil) 
teedback;  the  student  is  able 
locontirm  results  and  sirll-adjust 
as  needed 

The  iixscssim-iit  is  desipiuxi  iu*l 
mereh'  toaiidtl  pertormani e but  to 
imprnve  hiture  yiortoniialice  1 he 
student  IS  seen  as  the  prunarv 
"ruslornet^'  ol  inloi  malion. 
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Table  12.3  Bloom’s  Ranking  of  Thinking  Skills  (Bloom,  1956,  p.  100) 


Bloom’s  Ranking  of  Thinking  Skills 


Knowledge 

Comprehension 

Application 

Analysis 

Synthesis 

Evaluation 

List 

Name 

Identify 

Show 

Define 

Recognize 

Recall 

State 

Visualize 

Summarize 

Explain 

Interpret 

Describe 

Compare 

Paraphrase 

Differentiate 

Demonstrate 

Classify 

Solve 

Illustrate 

Calculate 

Use 

Interpret 

Relate 

Manipulate 

Apply 

Modify 

Analyze 

Organize 

Deduce 

Contrast 

Compare 

Distinguish 

Discuss 

Plan 

Devise 

Design 

Hypothesize 

Support 

Schematize 

Write 

Report 

Justify 

Evaluate 

Choose 

Estimate 

Judge 

Defend 

Criticize 
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Table  12.4  Taxonomy  Table  (Anderson  & Krathwohl,  2001,  p.  100) 


The  Cognitive  process  dimension 

The 

KNOWLEDGE 

Dimension 

1. 

Remember 

2. 

Understand 

3. 

APPLY 

4. 

ANALYZE 

S. 

Evaluate 

6. 

CRKATC 

A. 

FACTUAL 

KNOWLCDCC 

B. 

Conceptual 

KNOWLEDGE 

c. 

PROCeOURAL 

KNOWLCOGC 

D. 

MCTA- 

COGNITIVE 

KNOWLEDGE 
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Major  Types  and  Subtypes 


EXAMPLES 


A.  factual  knowledge — ^ThebiiMc  L'lCTnents  ^tiulwits  nui>5  know  to  IvacquAiiiL^  u1tli  a 

discipline oi  si'Kt  pioblems  ir  it 


Aa.  Knowlevl^f  ot  tYrminoli>>;v 
Ao.  Kn»m  Ifdjjjt' ot ‘ipt'iiH.  dotails  and 
demenU 


Tccrtnical  vocal'iiUrv,  musiral  symbol? 
Mjiiir  ualuiai  rtMmrcos,  idiablc  sources,  ul 
infyrmatiun 


B .  CONC.KPTU  AL  KNOWLKOOB — The  intertebtioiislTipb  .ir^ong  «lic  L-.isie  lunts  witliin  a Ltr-jor 

stniciiiiv  that  enable  them  to  iunelion  lutretlier 


Ba.  Kniwlr'd;^!  of  da‘"Sitii  ati(ins  and 

c.MCf;oriis? 

Ba  keinvitdge  of  pnn.‘i]'>lHs  .me) 
>;eT't*rali/ations 

Be.  Knowledge  of  tiut'riis,  m.dds,  and 
structiux-i 


I’l-nod?  <>f  geological  time  lomis  oi  biisines? 
ownership 

Pvlliagorean  Iheoiom,  law  ol  supplv  anil  iteniiinJ 
Ilieory  ol  evolution,  structure  of  Congress 


C.  PPOccQURAL  knowledge — Htuv  lo dll  bOJtu'tlimg,  me.thi nih  nt  irujiiirv,  ami  cnleriu  fur  using 

skills  algonthms  '.echninues,  ainj  rnethr«l' 

Ca.  Knowleiige  nl  subierN^pecific  ‘kdls  and  'skills  iimiI  in  p.iintiiig  witli  vcatemilors 
algonlhntLs  wholt'-numlx-r  (llvisinn  ilgorithm 

Cb.  Knowledge  ot  subiect-specJlic  lethnu|ues<  Intervwwiiig  ttsbru*;iu-s,  M.ii-jifitiL  melliud 
and  iiietlu'cls 

Cc.  Know  ledge  ol  cnlcria  for  determirung  Cntena  used  to  dctennnv  vvlu-n  lo  ij*ply  a 

when  to  use  .ippropnali'  prnrixluii'.-  pn^LeilLire  iliv  'h  iiig  Nlew  ton's  vcconi.  law  criteria 

U'.tsl  Si  ludgp  the  leasihilitv  ol  using  a particular 
mcthoci  to  e?tim."te  business  rc>sts 


D,  Metacognitive  KNowuEDae — Kii. neleifge  ol  ci  igiutiori  in  v',etiei  al  as  well  as  owareness  and 

know  h’ltgc  of  one's  nven  ro^^tioii 


Da.  Strategii  knowli-dge 


Dd.  Knowledge  about  cognitive  tasks, 
iru hiding  approjiriauic'onteNtual  and 
conditicinal  know  ledge 
Dc.  Self-kill iw'lodgi 


ICnowiedge  of  outlining  as  a moans  of  capturing 
lJU'  stroi  lure  I il  a Oi'il  o'  sul  it\  i iiidtlel  ill  a lext- 
hivik,  cnins  Iwigp  nt  me  use  «>l  henristirv 

Knowledge  of  the  U-pe?  Of  tests  partlc.ilo,'  wac  I'.er'- 
adminisier  knowlcHlgc  of  ttu?  uogiutive  den'..uul> 
of  tliffe''cni  tasks 

Knowk*dgP  lli.iteritii)inTigi-ssi'vs  is  .|  fx'rsonaJ 
itretigili,  w bore ,1s  wr  ung  essais  is  a piirs'i'al  weak- 
less,  awareivcks  ol  ,ine  a . mr  rmowledge  leicl 


Figure  12.4  Major  Types  and  Subtypes  of  the  Knowledge  Dimension  (Anderson  & 
Krathwohl,  2001) 


The  Kolb  Learning  Cyele  is  also  a useful  tool.  Fry  et  al.  (2003)  define  experiential 
learning  as  “learning  by  doing”  (p.  14).  They  eredit  Kolb  for  the  most  popular  theory  of  learning 
from  experience.  Experiential  learning  is  based  on  the  notion  that  understanding  is  not  a fixed  or 
unchangeable  element  of  thought  but  is  formed  and  re-formed  through  experience,  “a  continuous 
process  often  represented  as  cyclical,  and,  being  based  on  experience,  implies  that  we  all  bring  to 
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learning  situations  our  own  ideas  and  beliefs  at  different  levels  of  elaboration”  (p.  14).  The  Kolb 
Learning  Cyele  requires  four  abilities  when  learning  is  sueeessful  (p.  15). 


Concrete  Experience 
CE 


Active  Experimentation 
AE 


Abstract 

Conceptualization 
AC 

Figure  12.5  Kolb  Learning  Cycle  (Fry  et  al.,  2003,  p.  15) 

For  our  purposes,  this  eyele  ean  be  used  as  a metrie,  just  as  the  Bloom’s  and  revised  Bloom’s 
taxonomies  can  be,  to  guide  the  planning,  execution,  and  evaluation  of  teaching  and  learning. 

This  approach  makes  it  possible  to  deepen  learning  and  broaden  understanding  and  application 
of  knowledge  across  contexts  and  produces  more  authentic  learning  and  deeper  learning. 

Assessment  as  Learning 

So  far,  we  have  mentioned  assessment  o/ student  learning,  the  dominant  type  in  schools  today. 
Now,  let  us  discuss  two  other  types,  assessment /or  learning  and  assessment  as  learning  (Table 
5).  Assessment /or  learning  requires  a fundamental  shift  from  summative  assessment  (evaluative 
judgment)  to  formative  (ongoing)  assessment,  where  descriptions  are  created  that  can  guide  or 
inform  the  next  stage  of  learning: 

[Tjeachers  collect  a wide  range  of  data  so  that  they  can  modify  the  learning  work  for  their 
students.  They  craft  assessment  tasks  that  open  a window  on  what  students  know  and  can 
do  already  and  use  the  insights  that  come  from  the  process  to  design  the  next  steps  in 
instruction... Marking  (scoring  or  grading)  is  not  designed  to  make  comparative 
judgments  among  the  students  but  to  highlight  each  student’s  strengths  and  weaknesses 
and  provide  them  with  feedback  that  will  further  their  [individual]  learning.  (Earl  et  al., 
2003,  p.  24) 

This  changes  the  teacher’s  role.  Teachers  are  central  in  the  assessment  of  learning,  but  in 
assessment /or  learning,  they  use  their  knowledge  of  the  student  and  the  learning  context  to 
determine  learning  needs.  The  timing  is  also  different  as  assessment /or  learning  happens 
throughout  learning,  usually  more  than  one  time,  not  at  the  end  as  with  assessment  o/ learning. 

In  assessment  for  learning,  teachers  interact  with  students  differently  and  record-keeping  is  more 


Reflective 

Observation 

RO 
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of  the  checklist,  rubric,  and  portfolio  type,  showing  a progression  of  student  learning  along  a 
continuum  (Table  5;  Figures  6 and  7). 

Assessment  as  learning  has  been  our  model,  for  we  believe  that  teachers  and  students 
perform  best  while  learning  and  that  assessment  activities  can  serve  as  the  learning  mechanism. 
When  assessment  is  effective,  the  students  (in  our  case  the  teachers  and  ultimately  their  students) 
become  self- motivated  and  do  not  wait  for  the  teacher  (our  program  leaders)  to  “judge”  whether 
their  answer  or  work  is  correct.  Rather,  reflection  and  the  construction  of  meaning  enable  them 
to  realize,  on  their  own  or  with  team  members,  when  they  do  not  understand  something.  They 
then  take  steps  to  figure  out  what  they  need  and  where  or  how  to  find  it.  Comparisons  between 
individual  teachers  or  students  are  no  longer  relevant;  rather  they  compare  their  own  growth  or 
progress  with  their  prior  knowledge,  skills,  or  abilities. 

When  working  with  teachers,  we  generally  made  performing  and  learning 
indistinguishable  from  each  other,  one  driving  the  other.  The  teachers  presented,  discussed,  and 
ultimately  owned  the  goals  or  standards  they  achieved.  Then  they  participated  in  learning 
activities  with  the  needed  integral  knowledge  and  skill  development.  As  they  learned,  they 
developed  new  products,  attitudes  and  behaviors,  and  processes,  or  performed  using  new 
strategies,  models,  techniques,  or  procedures.  They  were  both  learning  and  being  assessed, 
simultaneously. 

This  is  one  method  of  “curriculum  embedded  assessment”  where  the  traditional  test  at  the 
end  is  replaced  by  the  culminating  product  or  performance.  Ongoing  immediate  feedback  is 
critical  in  this  process,  but  it  can  be  informal  as  well  as  formal.  Performance-based  assessment  is 
usually  not  arbitrarily  separated  from  learning  and  is  less  artificial  than  tests,  which  are  merely 
indicators  of  potential  performance  ability  or  of  “valued  real-world  performances”  (Wolf  & 
Reardon  and  Linn  & Baker,  as  cited  in  National  Society  for  the  Study  of  Education,  1996,  pp. 
19-20,  85). 

Table  12.5  Features  of  Assessment  of,  for,  and  as  Learning 


Approach 

Purpose 

Reference  Points 

Ke)' Assessor 

Assessment  of 
Learning 

Iiidgments  about 
placement, 
promotion, 
credentials,  etc. 

Other  students 

Teacher 

Assessment  for 
[.earning 

Information  for 
teachers' 
instructional 
decisions 

External  standards 
or  expectations 

Personal  goals  atid 

Leacher 

Assessment  as 
Learning 

Self- monitoring  and 
self-conection  or 
adjustment 

external 

standards 

Student 
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Figure  12.6  Traditional  Assessment  Pyramid  (Marzano  et  al.,  1993,  pp.  26-27) 


Figure  12.7  Reconfigured  Assessment  Pyramid  (Marzano  et  al.,  1993,  pp.  26-27) 

Authentic  or  Performance-Based  Assessment 

Standardized  tests  force  instruction  to  focus  on  the 

accumulation  of  facts  and  decontextualized  skills.  A very  different  approach  is  offered 
by  the  National  Council  of  Education  Standards  and  Testing,  the  National  Education 
Goals  Panel,  the  New  Standards  project,  the  American  Association  for  the  Advancement 
of  Science,  the  National  Councils  of  Teachers  of  English  and  Mathematics,  and  others. 
They  advocate  for  curriculum  reforms  that  emphasize  reasoning,  higher- order  thinking 
skills,  and  identification  and  solution  of  real-world  (authentic)  problems.  Therefore, 
authentic  performance-based  assessments  are  critical  and  integral  to  those  reforms  and 
considered  exemplary  instructional  practice  where  assessment  and  instruction  are 
indistinguishable  from  each  other.  (Einn  & Baker,  as  cited  in  National  Society  for  the 
Study  of  Education,  1996,  p.  85) 

The  National  Education  Goals  Panel,  in  its  first  report,  identifies  a need  for  higher 
standards  related  to  student  assessment.  Its  authors  advise  that  (authentic)  performance-based 


220 


assessments  may  be  more  closely  aligned  with  educational  goals  than  standardized  tests  (p.  86). 
They  (pp.  87-89)  provide  the  following  characteristics  for  performance-based  assessment: 

1.  Be  open-ended.  Require  the  student  to  construct  a response  or  perform  an  activity. 

2.  Involve  higher- order,  complex  skills.  These  would  include  formulating  and  solving 
problems,  reasoning,  and  communication. 

3.  Require  extended  periods  of  time  for  performance.  Include  the  collection  and  analysis 
of  data  as  well  as  the  preparation  of  written  or  oral  presentations  of  results  and 
conclusions. 

4.  Involve  group  performance.  Students  working  together  may  be  asked  to  formulate 
hypotheses  and  design  experiments. 

5.  Give  student  and  teacher  a choice  of  tasks.  Performance-based  assessments  often 
allow  some  degree  of  latitude  in  the  choice  of  tasks. 

6.  Rely  on  judgmental  scoring.  This  requires  scoring  guidelines  or  rubrics  and  training. 

In  addition,  there  is  much  to  be  considered  if  the  desire  is  to  design  valid  and  reliable 
performance  tasks.  Content  quality,  curricular  importance,  cognitive  complexity,  linguistic 
appropriateness,  ancillary  skills,  and  the  meaningfulness  of  the  tasks  for  students  are  all 
important  considerations  in  the  development  of  performance  tasks.  Also  important  are  the 
consequences  for  students  and  teachers,  fairness  when  assessing  diverse  learning  groups,  the 
transferability  of  results,  how  to  compare  results  over  time  with  confounding  cohort  differences, 
and  instructional  sensitivity  to  the  general  intellectual  ability  of  the  students.  And  finally, 
technical  quality  is  a definite  requirement  for  performance  assessments  (Linn  & Baker,  as  cited 
in  National  Society  for  the  Study  of  Education,  1996;  McMillan  et  ah,  2001). 

Wiggins  (1998,  pp.  139- 140)  establishes  that  performance  tasks  should  be: 

• Authentic,  they  should  have  realistic  options,  constraints,  criteria,  standards,  and 
audience,  and  a genuine  purpose. 

• Credible:  they  should  address  rigorous  content  and  performance  standards. 

• User-friendly:  they  should  be  feasible,  appropriate,  engaging,  and  rich  in  feedback, 
with  rehearsal  and  revision  built  in.  They  should  provide  a clear  and  complete  set  of 
instructions,  guidelines,  and  models;  and  troubleshooting  should  be  available. 

Wiggins  is  careful  to  warn  readers  that  while  performance-based  and  authentic  do  not 
mean  the  same  thing,  they  are  often  used  interchangeably  (pp.  140-141): 

1.  Just  because  it  is  a performance  task  does  not  mean  that  it  is  authentic.  Not  all 
performance  tasks  reflect  the  real  world. 

2.  A task  involving  hands-on  work  is  not  necessarily  authentic.  It  may  not  involve  the 
methods  and  procedures  of  the  real  world,  or  perhaps  the  answers  are  already  known. 

3.  A constructed- response  task  may  not  be  authentic  or  even  a performance  task. 
Performance  or  production  requires  the  student  to  plan  and  execute  a new  work  from 
scratch  and  to  use  good  judgment  in  choosing  content  and  shaping  a product. 
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4.  A task  that  is  authentic  is  not  necessarily  valid.  Validity  entails  the  ability  to  infer 
about  performance  based  on  an  apt  sample  of  work.  Many  authentic  challenges  make 
poor  assessment  tools  because  they  do  not  permit  easy  isolation  of  performance. 

5.  Just  because  a task  is  inauthentic  does  not  make  it  invalid. 

To  support  teachers  in  their  development  of  high-quality  assessments  and  performance 
tasks,  we  provided  teachers  with  examples  and  gave  immediate  and  continuous  feedback  as  they 
designed  and  developed  their  performance  tasks.  Wiggins  (1998)  has  easy-to-use  guidelines;  we 
also  used  Hart  (1994). 

Scoring  and  Grading 

Rubrics  make  public  the  key  criteria  that  students  use  in  developing,  judging,  and  revising  their 
work.  Rubrics  hold  both  the  student  and  teacher  accountable.  Students  know  and  understand 
what  they  have  to  do  to  achieve  at  established  levels,  and  teachers  cannot  “change  the  rules” 
once  the  rubrics  are  in  the  hands  of  the  students.  Rubrics  also  build  consistency  in  scoring  or 
grading,  while  reducing  bias.  And  most  importantly,  students  are  rarely  surprised  by  their  scores 
or  grades  (Huba  & Freed,  2000). 

Teachers  enthusiastically  reported  that  using  rubrics  resulted  in  higher- quality  student 
work,  more  learning,  improved  scoring  or  grading,  and  better  teacher- student  relationships. 

They  also  felt  that  the  self-assessment  integral  to  the  use  of  rubrics  motivated  students  to  achieve 
at  higher  levels. 

There  are  all  kinds  of  rubrics.  We  developed  very  simple  ones  for  use  in  scoring  the 
teachers’  performance  tasks,  rubrics,  and,  ultimately,  curricula  modules.  In  considering  what 
good  rubrics  need,  the  following  elements  are  important: 

• Levels  of  mastery  describe  the  level  the  student  has  achieved. 

• Dimensions  of  quality  can  be  discipline  specific  or  include  general  education 
knowledge  and  skills,  cognitive  processes,  procedural  knowledge,  etc. 

• Organizational  groupings  are  clusters  of  criteria  within  a particular  grouping. 

• Commentaries  are  the  descriptions  of  each  criterion  by  level. 

• Description  of  consequences  is  where  the  teacher  reveals  the  consequences  of 
performing  at  a given  quality  level  in  a real-life  setting.  A form  of  feedback 
encourages  students  to  think  about  what  will  happen  in  a real-world  setting.  (Huba  & 
Freed,  2000,  p.  167) 

Rubrics  can  broadly  accomplish  two  important  things:  educate  students  and  judge  their 
work.  Students  learn  the  expected  or  optional  standards  of  the  discipline  or  profession, 
internalize  them,  and  build  aspirations  for  themselves,  connecting  what  they  are  learning  to  their 
real  world  after  graduation.  Students  also  become  informed  about  what  constitutes  poor,  good, 
and  excellent  qualities  of  work  or  performance.  They  can  use  this  to  self-assess,  provide 
themselves  with  feedback,  and  then  correct  their  work  before  turning  it  in  to  the  teacher. 

Teachers  asked  us  for  benchmarks  or  work  that  exemplified  qualities  of  work  that  we  expected 
of  them.  Huba  and  Freed  (2000)  have  suggested  that  students  examine  “ungraded”  work  across 
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various  levels  of  quality:  “When  students  have  an  opportunity  to  examine  assignments  that  differ 
in  quality,  they  usually  find  that  their  own  work  is  enhaneed”  (p.  170). 

There  are  a variety  of  styles  of  rubries,  some  with  deseriptions  as  in  the  samples  provided 
below,  and  others  with  more  simply  stated  criteria  below  each  standard  (Figures  8 through  14  in 
the  appendix;  also  see  examples  on  the  website,  www.strategicalliance.niu.edu.). 

Rubrics  must  have  prescribed  standards  and  criteria  levels  for  each  standard,  establishing 
various  levels  of  achievement  possible  with  a point  system.  Standards  can  be  absolute  or 
developmental.  The  absolutes  are  those  exemplary  benchmarks  established  de  facto  as  the  best. 
Developmental  standards  are  established  for  particular  cohorts  (e.g.,  high  school  versus  college), 
so  excellence  is  determined  for  each  type  of  cohort.  Expectations  are  different  in  nature. 
Expectations  reflect  patterns  or  norms  for  groups;  in  other  words,  a student  is  expected  to 
perform  at  a particular  level  because  there  is  a pattern  of  that  result  for  his  or  her  ability  and 
experience  levels.  “Students  can  exceed  norms  and  expectations  but  still  not  perform  up  to 
standard”  because  norms  have  the  effect  of  “hiding  how  students  and  teachers  are  doing  when 
judged  against  worthy  standards”  (Wiggins,  1998,  pp.  157-158).  Good  rubrics  have  a logic  and 
chronology.  They  also,  as  with  performance  tasks,  need  to  be  valid  in  that  the  appropriateness 
and  validity  of  the  criteria  and  descriptors  for  discrimination,  or  making  judgments  against,  are 
valid  in  relation  to  the  tasks.  These  are  holistic  and  analytic  trait  rubrics.  Criteria  are  often  of 
five  types:  impact,  craftsmanship,  methods,  content,  and  sophistication  of  performance.  One 
need  not  use  all  five  all  the  time;  “the  challenge  is  to  make  sure  that  we  have  a feasible  set  of 
right  criteria  and  that  we  have  distinguished  between  genuine  criteria  and  mere  indicators  or 
useful  behaviors”  (p.  168).  Weights  are  important  considerations.  Eor  example,  should 
processes  and  results  be  weighted  the  same  or  should  the  mechanics  of  writing  and  the  content  be 
equally  weighted?  Rubric  descriptors  should  address  both  strengths  and  errors  in  the  work  or 
performance  j udged. 

Summing  up  Rubrics 

According  to  Wiggins  (1998,  pp.  184-185),  the  best  rubrics 

1 . Are  sufficiently  generic  to  relate  to  general  goals  beyond  an  individual  performance 
task,  but  are  specific  enough  to  enable  useful  and  sound  inferences  about  the  task. 

2.  Discriminate  among  performances  validly,  not  arbitrarily,  by  assessing  the  central 
features  of  performance,  not  those  that  are  easiest  to  see,  count,  or  score. 

3.  Do  not  combine  independent  criteria  in  one  rubric. 

4.  Are  based  on  analysis  of  many  work  samples  and  on  the  widest  possible  range  of 
work  samples,  including  valid  exemplars. 

5.  Rely  on  descriptive  language  (what  quality  or  its  absence  looks  like)  as  opposed  to 
merely  comparative  or  evaluative  language,  such  as  “not  as  thorough  as”  or 
“excellent  product,”  to  make  a discrimination. 

6.  Provide  useful  and  apt  discrimination  that  enables  sufficiently  fine  judgments,  but  do 
not  use  so  many  points  on  the  scale  (typically  more  than  six)  that  reliability  is 
threatened. 
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7.  Use  descriptors  that  are  sufficiently  rich  to  enable  student  performers  to  verify  their 
scores,  accurately  self-assess,  and  self-correct.  (Use  of  indicators  makes  description 
less  ambiguous,  hence  more  reliable,  by  providing  examples  of  what  to  recognize  in 
each  level  of  performance.) 

8.  Highlight  judging  the  impact  of  performance  (the  effect,  given  the  purpose)  rather 
than  over-rewarding  processes,  formats,  content,  or  the  good- faith  effort  made. 

According  to  Wiggins,  technical  rubrics  are 

1.  Continuous.  The  change  in  quality  from  score  point  to  score  point  is  equal:  the  degree 
of  difference  between  a 5 and  a 4 is  the  same  as  between  a 2 and  a 1.  The  descriptors 
reflect  this  continuity. 

2.  Parallel.  Each  descriptor  parallels  all  the  others  in  terms  of  the  criteria  language  used 
in  each  sentence. 

3.  Coherent.  The  rubric  focuses  on  the  same  criteria  throughout.  Although  the  descriptor 
for  each  scale  point  is  different  from  the  ones  before  and  after,  the  changes  concern 
variance  of  quality  for  the  (fixed)  criteria,  not  language  that  explicitly  or  implicitly 
introduces  new  criteria  or  shifts  the  importance  of  the  various  criteria. 

4.  Aptly  weighted.  When  multiple  rubrics  are  used  to  assess  one  event,  there  is  an  apt, 
not  arbitrary,  weighting  of  each  criterion  in  reference  to  the  others. 

5.  Valid.  The  rubric  permits  valid  inferences  about  performance  to  the  degree  that  what 
is  scored  is  what  is  central  to  performance,  not  what  is  merely  easy  to  see  and  score. 
The  proposed  differences  in  quality  should  reflect  task  analysis  and  be  based  on 
samples  of  work  across  the  full  range  of  performance;  describe  qualitative,  not 
quantitative,  differences  in  performance;  and  not  confuse  merely  correlative 
behaviors  with  actual  authentic  criteria. 

6.  Reliable.  The  rubric  enables  consistent  scoring  across  judges  and  time.  Rubrics  allow 
reliable  scoring  to  the  degree  that  evaluative  language  (“excellent,”  “poor”)  and 
comparative  language  (“better  than,”  “worse  than”)  is  transformed  into  highly 
descriptive  language  that  helps  judges  to  recognize  the  salient  and  distinctive  features 
of  each  level  of  performance. 

Student  assessment  fits  within  the  broad  context  of  instructional  design.  The  process 
begins  with  an  instructional  analysis  to  determine  what  teachers  think  they  are  doing,  what  they 
should  be  doing,  their  strengths  and  weaknesses,  and  what  they  think  they  are  teaching  and 
measuring.  We  used  the  learning  standards  as  part  of  this  process  and  helped  teachers  focus  on 
what  the  state  and  nation  prioritize  for  students  to  learn.  We  arranged  this  process  in  a variety  of 
ways.  Usually,  we  engaged  teachers  in  the  analysis  at  the  beginning  of  the  professional 
development  program.  Later,  they  followed  through  when  participating  in  the  performance 
assessment  component.  We  modeled  best  practice  in  teaching  and  learning,  and  used  standards, 
performance  assessment,  feedback,  and  rubrics  to  score  the  results.  The  workshop  leader  worked 
with  teachers  in  a reiterative  process  that  required  a great  deal  of  patience. 
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To  support  our  teachers  in  this  process,  we  gave  them  reference  copies  of  several 
different  sources.  These  varied  according  to  the  year  and  included  the  following:  Marzano  et  al. 
(1993),  Assessing  Student  Outcomes,  Performance  Assessment  Using  the  Dimensions  of 
Learning  Model;  McTighe  and  Arter  (2001),  Scoring  Rubrics  in  the  Classroom;  Hart  (1994), 
Authentic  Assessment;  Burke  (1999)  How  to  Assess  Authentic  Learning;  and  Bellanca  et  al. 
(1994),  Multiple  Assessments  for  Multiple  Intelligences. 

Developing  teachers  to  the  point  where  they  can  devise  reasonably  good  and  reliable 
performance  tasks  and  rubrics  means  more  than  just  teaching  them  the  process;  it  requires 
embedding  performance  tasks  within  real-world  problems  and  scenarios.  Making  this  connection 
is  very  difficult  for  most  teachers,  especially  those  in  mathematics  and  the  sciences.  It  is  much 
less  difficult,  and  in  fact  more  normal,  for  those  in  vocational  areas  or  English  disciplines. 
Therefore,  MSTE  teams  benefited  from  working  together  on  developing  performance  tasks  and 
rubrics.  Vocational  teachers  were  a great  asset  when  guiding  mathematics,  science,  and  English 
teachers  to  use  and  develop  these  more  authentic  and  performance-based  kinds  of  assessments 
(Tables  6,  7,  and  8). 
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Table  12.6  Rubric  for  Assessing  the  Quality  of  a Performance  Task 

Key  Components 

A properly  designed  Performance  Task  must: 

a.  be  based  on  content  standards  taken  from  the  Illinois  Learning  Standards 

b.  describe  a “real-life”  scenario 

c.  involve  students  in  complex  reasoning  processes 

d.  require  students  to  collect  and  process  information 

e.  incorporate  “habits  of  mind” 

f.  require  student  collaboration  and  cooperation 

g.  result  in  a tangible  product  or  communication  activity 

I.  Component:  The  Performance  Task  is  based  on  the  Illinois  Learning  Standards. 

1.  The  Performance  Task  is  directly  related  to  and  based  on  Learning  Standards. 

2.  Learning  Standards  are  apparent,  but  the  relation  to  the  task  is  sketchy  or  irrelevant. 

3.  The  Performance  Task  does  not  appear  to  be  based  on  Learning  Standards. 

II.  Component:  A “Real-Life”  scenario  is  described  in  the  Performance  Task. 

1.  The  scenario  described  in  the  task  accurately  mirrors  an  activity  in  the  community 
outside  the  classroom. 

2.  The  scenario  described  in  the  task  simulates  an  activity  in  the  community  outside  the 
classroom. 

3.  The  scenario  described  in  the  task  contains  some  aspects  of  activity  outside  the 
classroom  but  is  largely  contrived. 

4.  The  scenario  described  in  the  task  is  an  academic  exercise  that  usually  takes  place 
only  in  the  context  of  a school  setting. 

III.  Component:  The  Performance  Task  involves  students  in  complex  reasoning  processes. 

1.  The  task  requires  students  to  utilize  complex  reasoning  components,  such  as 
induction/deduction,  diagnosis,  abstracting,  experimental  inquiry,  or  problem 
solving. 

2.  The  task  requires  students  to  utilize  complex  reasoning  components,  such  as 
comparing,  classifying,  decision  making,  or  investigation. 

3.  The  task  requires  students  to  only  recall  facts. 

rV.  Component:  The  Performance  Task  requires  students  to  collect  and  process  information. 

1.  The  task  incorporates  a variety  of  information  gathering  techniques  and  information 
resources.  Students  are  required  to  interpret  and  synthesize  information  and 
accurately  assess  the  value  of  information  gathered. 

2.  The  task  requires  students  to  gather  and  synthesize  information,  but  the  value  of  the 
information  gathered  is  not  assessed. 

3.  The  task  requires  students  to  gather  information  but  not  to  interpret  it. 

4.  The  task  requires  no  gathering  or  processing  of  information. 


226 


Table  12.7  Rubric  for  Assessing  the  Quality  of  a Performance  Task  (continued) 

V.  Component:  The  Performance  Task  incorporates  “Habits  of  Mind.” 

1.  The  task  requires  students  to  make  effective  plans,  use  necessary  resources,  evaluate 
effectiveness  of  own  actions,  seek  accuracy,  and  engage  in  activities  when  answers  or 
solutions  are  not  immediately  apparent. 

2.  The  task  only  requires  students  to  effectively  plan  or  use  resources. 

3.  The  task  does  not  require  students  to  engage  in  self -regulation,  critical,  or  creative 
thinking. 

VI.  Component:  The  Performance  Task  requires  student  collaboration  and  cooperation. 

1.  The  task  requires  students  to  use  interpersonal  skills,  work  toward  the  achievement  of 
team  goals,  and  perform  a variety  of  roles  within  the  team. 

2.  The  task  requires  students  to  work  together  in  hams,  but  there  are  no  measures 
described  that  ensure  collaboration  or  cooperation  among  team  members. 

3.  The  task  is  completed  largely  by  students  on  an  individual  basis  rather  than  in  student 
teams. 

VII.  Component:  The  Performance  Task  results  in  a tangible  product  or  communication  activity. 

1.  The  task  result  is  a tangible  product  or  communication  activity  comparable  to  that 
commonly  produced  in  business  or  industry. 

2.  The  task  results  in  a product  that  is  similar  to  those  completed  in  business  or  industry 
but  lacks  several  components  that  make  the  product  realistic. 

3.  The  task  does  not  result  in  a product  or  communication  activity. 
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Table  12.8  Rubric  for  Assessing  the  Quality  of  a Rubric 

Key  Components 

A properly  designed  Rubric  must: 

a.  Contain  a set  of  key  components  to  be  assessed. 

b.  Include  descriptors  for  each  key  component. 

c.  Have  descriptors  that  are  indicative  of  observable  student  performance. 

d.  Include  appropriate  weights  for  each  component  and  descriptor  (optional). 

I.  Component:  The  Rubric  contains  a set  of  key  components  to  be  assessed. 

Level  Descriptors 

1 A complete  list  of  key  components  is  provided. 

2 Key  components  listed  are  not  exhaustive  for  the  performance  task. 

3 Not  all  key  components  describe  student  outcomes. 

4 No  key  components  are  listed. 

n.  Component:  The  Rubric  includes  a set  of  descriptors  for  each  key  component. 

Level  Descriptors 

1 Descriptors  for  each  component  are  arranged  in  a clear  hierarchy  from  non-achievement 
to  full-achievement. 

2 Descriptors  are  present  for  each  component,  but  obvious  levels  in  some  are  missing. 

3 Each  component  does  not  have  an  associated  set  of  descriptors. 

III.  Component:  Rubric  descriptors  are  clear  and  contain  observable  student  behavior. 

Level  Descriptors 

1 All  descriptors  clearly  delineate  levels  of  student  performance. 

2 Most  descriptors  clearly  delineate  levels  of  student  performance. 

3 Only  a few  descriptors  clearly  define  levels  of  student  performance. 

4 Descriptors  do  not  describe  observable  student  outcomes. 

IV.  Optional  Component:  Appropriate  weights  are  assigned  to  components  and  descriptors. 

Level  Descriptors 

1 Components  and  descriptors  are  each  properly  weighed  according  to  instmctional 
emphasis. 

2 Weights  are  assigned,  but  point  values  do  not  reflect  proper  instructional  emphasis  in  all 
cases. 

3  Weights  are  assigned  to  some  performance  standards  and  descriptors. 

Our  teachers  had  already  identified  the  student  learning  standards  upon  which  they  were 
going  to  base  their  integrated  MSTE  modules  and  had  spent  time  with  businesses  and  industries 
related  directly  to  their  modules.  We  began  by  reviewing  their  instructional  analyses,  the 
standards  they  wanted  students  to  achieve,  and  then  working  with  them  on  the  development  of 
the  pretest/posttests,  followed  by  the  development  of  performance  tasks.  The  performance  tasks 
were  embedded  within  real-  world  problems  or  scenarios,  using  their  business,  industry,  and 
community  learning  experiences  as  a basis.  After  that,  they  designed  their  assessment 
instruments  or  procedures,  including  their  tasks  and  rubrics,  so  that  each  module  had  several 
complex  performance  tasks  (cluster  of  tasks)  and  corresponding  rubrics.  As  mentioned  above, 
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they  had  also  developed  a traditional  pretest  and  posttest  that  was  approved  by  the  program 
leader  and  assessment  expert. 

The  workshop  covered  assessment  broadly,  and  then  each  type  of  assessment  was 
explored  more  deeply,  focusing  on  more  authentic  performance  assessments,  the  scoring  of 
performance  tasks,  and  the  creation  of  rubrics.  The  teachers  developed  the  module’s  pretest  and 
posttest  with  close  assistance  from  the  program  leader.  Bloom’s  Taxonomy  was  reviewed  and 
used  as  a tool  in  the  development  process  for  all  types  of  assessment.  At  each  stage  of  this 
usually  week-long  course  or  workshop,  the  teachers  developed  assessment-related  tools,  so  they 
were  learning  while  performing  and  being  simultaneously  assessed  through  an  interactive 
process  of  teaching,  development,  feedback,  and  evaluation  between  the  leader  (professor)  and 
teacher. 

This  process  was  most  effective  when  done  over  a four-day  span,  but  we  also  tried  it  for 
five  days  and  three  days.  Five  days  ensured  greater  understanding  and  skill  development;  three 
days  was  too  short.  Here  is  the  agenda  for  a four-  or  five-day  workshop: 

Day  1:  Assessment  Overview.  Goal:  Gain  literacy  in  assessment  and  assessment  strategies; 
identify  the  components  of  a balanced  assessment  model. 

Day  2:  Portfolio  Assessment.  Goal:  Identify  the  components  and  uses  of  portfolios;  design  a 
portfolio  for  the  assessment  workshop. 

Day  3:  Traditional  Assessment.  Goal:  Design  teacher- made  tests  that  are  aimed  at  higher 
cognitive  levels;  build  a database  of  test  items  to  measure  learning  benchmarks  for 
each  content  area;  develop  a team  instrument  to  assess  student  accomplishment  of 
module  objectives  (standards). 

Day  4:  Performance  Assessment.  Goal:  Identify  assessment  tools;  design  performance  tasks. 

Day  5:  Designing  Rubrics.  Goal:  Develop  skills  in  writing  and  using  rubrics;  design  rubrics 
to  assess  student  accomplishment  of  performance  tasks. 

Teachers  received  reference  texts,  a notebook  with  information  from  the  presentations, 
performance  task  and  rubric  examples,  the  standards  and  achievement  criteria  for  each  one,  and  a 
rubric  for  scoring  their  performance  tasks  and  rubrics.  The  leader  reviewed  the  traditional  tests 
for  standards-based  content  validity.  Teachers  were  also  required  to  keep  a reflective  journal  of 
their  development,  which  had  prompts  for  their  responses.  They  produced  an  informal  mini- 
portfolio, which  provided  evidence  of  their  growth  in  student  performance  assessment  and 
traditional  test  development. 

Teachers  responded  extremely  well  when  the  workshops  were  led  by  a knowledgeable 
person  who  could  answer  their  questions  and  help  them  make  learning  and  assessment  real.  This 
required  a leader  with  experience  beyond  a traditional  classroom  and  who  could  use  mathematics 
and  science  across  real-world  and  other  learning  contexts.  The  most  successful  leaders  came 
from  technical  disciplines  (e.g.,  industrial  technology,  engineering  technology).  We  had  great 
success  with  teams  co-  led  by  a professor  from  engineering  technology  and  a high  school  master 
teacher  from  technology  education.  However,  the  workshops  were  less  successful  when  the 
program  transitioned  to  the  district,  where  local  leaders  took  over.  The  traditional  tests  were  not 
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as  well  conceived  or  developed,  and  the  performance  tasks  were  less  authentic  and  much  more 

basic,  primarily  due  to  lack  of  experience  outside  the  classroom  or  with  theory  in  action. 
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Appendix 


congtraip^h  oi- obstacles' 

4 Accurately  and  thoroughly  describes  the  relevant  constraints  or  obstacles. 

Addresses  obstacles  or  constraints  that  are  not  immediately  apparent. 

3 Accurately  identities  the  most  important  constraints  or  obstacles 

2 Identifies  some  constraints  or  obstacles  that  are  accurate  along  with  some  that  are 
not  accurate. 

1 Omits  the  most  significant  constraints  or  obstacles. 

Identifies  viable  and  important  alternatives  for  overcoming  the  constraints  or 

obstacles. 

4 Identifies  creative  but  plausible  solutions  to  the  problem  under  consideration. 

The  solutions  address  thu-  central  difficulties  posed  by  the  constraint  or  obstacle 

3 Proposes  alternative  solutions  that  appear  plausible  and  that  address  the  most 
important  constraints  or  obstacles 

2 Presents  alternative  solutions  for  dealing  with  the  obstacles  or  constraints,  but  tht 
solutions  do  not  all  address  the  Important  difficulties. 

1 Presents  solutions  that  faU  to  address  critical  parts  of  the  problem. 

Selects  and  adeijuatehf  tries  out  alternatives. 

4 Engages  in  effecti\  e,  valid,  and  exhaustive  trials  of  the  selected  alternatives. 

Trials  go  beyond  those  required  to  solve  the  problem  and  show  a commitment  to 
an  in-depth  understanding  of  the  problem. 

3 Puts  the  selected  alternatives  to  trials  adequate  to  determine  theu-  utility. 

2 Tries  out  the  alternatives,  but  the  trials  are  incomplete  and  important  element^ 
are  omitted  or  ignored. 

1 Does  not  satisfactorily  test  the  selected  solutions. 

If  other  alternatives  were  tried,  accurately  articulates  and  supports  the  reasoning^ 

behind  the  order  of  their  selection  and  the  extent  to  which  each  overcame  the 

obstacles  or  constraints. 

4 Provides  a clear,  comprehensive  summary  of  the  reasoning  that  led  to  the  selec- 
tion of  secondary  solutions.  The  description  includes  a review  of  the  decisions 
that  produced  the  order  of  selection  and  how  each  alternative  fared  as  a solution. 

3 Describes  the  process  that  led  to  the  ordering  of  secondarv’  solutions.  The 
description  offers  a clear,  defensible  rationale  for  the  ordering  of  the  alternatives 
and  the  fmal  selection, 

2 Describes  the  process  that  led  to  the  ordermg  of  secondary  solutions.  The 
description  does  not  provide  a clear  rationale  for  the  ordering  of  the  alternatives, 
or  the  student  does  not  address  all  the  alternatives  that  were  tried. 

1 Describes  an  illogical  method  for  determbiing  the  relativ’e  value  of  the  alternatives. 

The  student  does  not  present  a reasonable  review  of  the  strengths  and 
w'^eaknesses  of  the  alternative  solutions  that  were  tried  and  abandoned 

tSource:  McREL  Institute)  ' 

Figure  12.8  Problem-Solving  Rubric  (McRel  Institute  as  cited  in  Huba  & Freed,  2000,  p. 
191) 
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fs  axvfit'e  of  own  thinking. 

4 Conii^.tently  .^nd  df  curdtelv  explains  in  dcl.iil  the  sexjuence  of  llmughts  he  or  site 
uses  when  fared  with  a task  nr  problcai,  and  provides  analyses  of  how  ait 
awareness  of  rtWn  thinking  has  onJianced  his  itr  her  performance. 

3 Consistently  and  accurately  describes  hotv  he  or  she  Dunks  thrcmgli  tasks  or 
problems  and  how  an  awareness  of  own  thinking  enhances  hi.s  or  her  performance. 

2 Sporadically  but  accirrately  dt*scnbes  how  lie  ot  she  thinks  through  tasks  or 
problems  and  how  an  awareness  of  own  thinking  enitanccs  his  or  her 
performance. 

I Rarely,  if  et  er,  accurately  describes  how  he  ot  she  thinks  through  tasks  or 
problems  or  how/  an  awareness  of  his  or  her  Dtinking  enhances  performance 

Is  open-minded. 

4 C onsistently  seeks  out  different  and  opposing  points  of  view  and  considers 
altemahve  views  impartially  and  rationally 

1 Is  ron.sistently  aware  of  points  of  view  that  differ  from  hts  or  her  own  and  always 
makes  a concerted  effort  to  considci  alternative  views 

2 Is  at  times  aware  of  points  of  view  that  differ  from  his  or  her  omi  and 
sporadically  makes  an  citori  to  consider  alternative  view.s 

1 Rarely,  if  ever,  is  aware  of  points  of  view  that  differ  from  Ills  or  her  own  and 
seldom  makes  an  effort  to  consider  altemativ'e  views. 

Restrains  impulsivHy. 

4 (Jonsisfcntly  and  carefully  considers  situations  to  delonnine  if  more  study  is 
required  before  acting;  when  further  studyis  required,  engages  in  detailed 
rnvestigdhon  before  acting. 

3 Consistently  considers  situations  to  determme  whether  more  study  w rec[uucd 
before  acting,  when  further  studyis  requiicd,  gathers  sufficient  information 
before  acting. 

2 Sporadically  considers  .situations  to  determme  w'hcther  more  study  is  required 
before  acting;  when  further  study'  is  icquued,  .somehmes  gathers  sufficient 
information  before  ac  ting 

1 Rarely,  if  ever,  considers  situations  to  determine  W'hethcr  more  study  is  required 
before  acbng;  when  further  study  is  required,  usually  doesn't  gathei  sufficient 
information  before  acting 


(Souiver  McREL  Institute) 

Figure  12.9  Habits  of  Mind  Rubric  (McRel  Institute  as  cited  in  Huba  & Freed,  2000,  p.  192) 
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L«v«U  uf  Achiev«>n«nl 

Crifwia 

E.x.cel1«Mt  (A) 
4 piHnt*> 

Good  (B) 
3 points 

Needs  Improvement  1C,  Dl 
2 |toLnts 

UTUccepiiblv  tl'l 
1 point 

t\ymvUt:ctr  <Jt 

J cmuidtinn  and 
HO,»rt*  pr»»bJerti 

Design  f nibleer  hmrt* 

uI.itkiHi  mclisir 

AeUthi'>U|;Ut  ':>iiC  IHc 
probien'  ,m.i^  is  wHl 
Jefinni. 

Ttu  probieni  forrTU'I.MV*P 
in  c3tMr  H'lt  tlic  »S 

n»t  well  dk'fuVed 

f)ie  pri  ihleci  loni  lUl  JtKn  • 
u ufU’k'ar  in  «rtwere*<pm’t-- 
)nd  dors  tv><  appear  tj*  tv 
well  llu'ught  ivd 

Fhk'  ilfsigp  ptohkrr  is 
mil  fi>  TvuiiaMli*lf^»r\ 

The  pfx»6lrm  ; hi:f*en 

d i uirt^it 

^liallrnge  foc*ng  the 
vn>tioi*  indusiry  The 
f'«Xennal  »nnrkti  ks  Urg* 
.ind  ctortv  KlrnhhrJ. 

Ihi*  pTii^iWm  n?pn>cntv  a 
cunt^f  itviHengc  in  the 

en^ru'  tntlustii'V  . i>«;l  ttu* 
pitenh.il  tndrki'l  is  small 
II  )N  md  iilintified 

1 he  pmblem  di>Cs  iiol 
ri-prbwerit  a i tmeiu 
.halU'ngc  ai  the  engine 

industrr.  -ind  lh«*  m.^rket 

is  -Ticilli^r  is  tvsKtrahv 
tdrf'tihrj. 

Pv  prohlm  cluw  not 
r<*prr?vr.r  i 

<lMlU-f'gL*  m tn<»  *>ngTTW 
iiiiJuativ  rheit  IS  na 

I'liplAiitihon  abovl 
>vnuKI  t*- MOeteHCvi  m 

ptiuiu4»  v.-h^  !hov 
t>H>‘  i!  n’t-rr  i«  i*.' 
evUiriK'r  Iht*  l»iV 
f;t.ajnJ  r^iurk  (»  g,,  ouirkrl 

\1psirh  .in  4‘ngine. 


iVmtpiNrJ 

Figure  12.10  Rubric  for  Engine  Design  Project  (Part  1)  (McRel  Institute  as  cited  in  Huba  & 
Freed,  2000,  p.  191) 


Levels  ul  Achlevemeiil 


CrilerU 

E^cellenl  lAl 
1 point* 

Gocidini 
1 points 

Need*  ImpiovetAenl  (C,  D) 
2 point* 

L'n«cc«*ptjblc  O'l 
I pOl.ll 

Ii/yiKVV-ri’iy  Sk  Jl 

Livgimvnrj; «ndl>s»s 
JeUiiled  ami  4rh*illengln^ 
and  is  uMsd  at  ei'crr  *l  igt 
:if  the  design  prucf*^ 

nv.'»'ngii>i*inr»|j  .moly il  - 
ls dotaikxl  ami  dvli«i^‘ni;, 
but  3* 'tie  lU*  f|i>4 

ipj’t’ar  li»  be  MtppofUxt 
hv  lakuidtUms. 

Somr  tuulvsii  u uvdudcdc 
hnl  tl  If-  rnt  very  drlatUsl 
r»i  clMih^g'nis*  klanv 
-•h'p^  dfx  ni»t  flipported 

b>  iaikUldm.»ns 

T-nginu'^nj^  ooilvsi-.  lb 
uitremiemfv  usvii  IVhi’n 

iMcd  it  jpptJrs  lh%  l.d 

.snd  liMil*.  tn  nhvimis 
ivn-  lu>inn-.. 

L)ix  amr^tatitm 

DiH  unvnutiun  is 
Uirifiiugh  anil 
conipK'tr 

Tlieir  is si'nv  mi'-sing  m 
iiitm<ii»nn  in  th»’s«iK*» 
menLltlnii 

fhiTT  IS  a gn*Jl  iitSAl  0^ 
imwirig  1 nf nrmatii'n  in  the 
wlis.unx’nlatiiyi 

OiMuncntiibiirt  isp^ic  ny 
pynr:  sis-teni 

.\sj,umptxnis 

All  d'i^i  imptons  arc  M.nr\| 
aixl  iu-sitfitsi. 

A^NumptniriH  jrv  stall'd 

hoi  -lime  are  not 

AfisumpMons  an*  sutyj 
bul  nime  are  fustifh.*sl 

Pmo  ,is4an*tjirum  are 
-.Uhil 

rit/wsifw  rf  K^nin'ira>(, 
ahnut  Inff^a! 

hi'Viind  in 

the  pn.'itNiuwiU:'  uHTNe 
are  tre^oenth  used  Tlii* 
pntte*.‘An  ma\  h.n'i 

Icoi'nc*-  -uiTK  thing  m*«v 

rr«*ivi|UAiU‘  lourse  •.■nnanit 
'•n  iijiisi  r.isih*  .mil  Mirre 
uidli’Yi  1 Ivs  nnJ  tin 
..*»ji!-e  - 

iVenvjii  s4b'»*H|fs»  lOiiti'iit 

K ail'd,  bin  III  i>  .11. il 
iint^frili»r  jn*a»  an*  n'>t 
ir’fnv/uivd 

qiiicitf  . .iu*sb- 
viM'lV'nt  not  ipptied 
Mirm'il^  \«*w  i»- 

tiOi  iitcJudi.'d 

Figure  12.11  Rubric  for  Engine  Design  Project  (Part  2)  (McRel  Institute  as  cited  in  Huba  & 
Freed,  2000,  p.  191) 
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C.n.>,ip  functioning 

rhpi^rvtip  Kjrt<^no*i 
Pnrr  'rvifv^  inJK«i)rs>cixKj 

jHiTibuiit^oi  Wi«n  Ml 

fiwirbevs  cUM  1 h-illongt%^ 
ii»J  i>e*  tbvu  MiftiTibir 
hllfVI  JfV  VoilicJ 

riw  Knntp  fuiWms  hir4> 
well  Soni.  inllH- 

>*njup  NebevfUM  f aiv 
wiekiii)^  (im  les- 

luiij/  liian  c'Uwt>s.  l>ui 
ovcn  ''fH*  f untTjhiihn^*. 

Tbr  gTtnij-  IS  -rth  ruiiilu>/i* 
Mip.  hut  rac+i  mdwnia,il 

drnnt  hjs.  VTMSvn  >^^^Tk 
and  •gntivmn  die  rttirts  i«J 

• •llvrs  Tins V .arc  Irc^urfd 

c'pi-stiiks,  v,'V‘V 

ivr>xin  >■  dod^  will  not  rjl 

iviUs  aniitfM.'r  Jtt«  t(  la«  U 

• >1  roiiiitiu:u«.uU4n. 

TV*  ftHHip  ivitvfiohs 
p^Mvlv  All 
ruRiuaot  Hui'  .idual 
el  tons 

Vi->;uUtit>  nO 
t'locfuctivm  ,>l 

fKv  pfoi.ip  mi*rls 
ind  iHi*  nmiMKH  atv 
pii  nbt<»tv*- 

rtu*  ^oup  rne^t-  rej»ubuiv 

bu./S'CetiM^;w  *r»-  init  w 
p^Hev  live  .is  CT3i»M 

te.  ScMTU.*  niC'llJUl- 4M;  I10» 
prepaivJ. 

Ih»-  gr  iMj'  nirt'U  imv>e 
Itfrly.  Mee»mf>  a»V  tiol  u- 
a«thrv  :iiii'd 
fphcciht&r  M’ver.il 
nvmtvrsiiTm*i  jTi'j.umi. 

t>€  dOCA  nut  fTUri 

iV|*lliarl\  ii'd  trbep 't 

v<«m*  ileutbiss  at» 
V *-s 

f'lefseed 

Um^oI 

vnivinri; 
tW  Hpsiqur> 

lilt*  ' 111.11.1*^  tin|ueri 

use  of  braiMKiutiMnc  jnd 
.^Kjup  pnil'IvM  -sols  ir< 
tcvlink)ui^enJ  Joi 
tisese 

Tin  t-i.Mf 

.inj  ^m  ip 

pror*li*ni  Mi|vtfi>|  trvlute 
•^luv  but  til  K*s  not  .Vv'nv*^ 
dorunient  tV-  eftev't  ot 
the'ie  s*.'Hv«7ns 

s«  111  It-  .itioiu|  I *M  iM  ^roup 
LifS't'Kii'-sd'.  ifit;  h<h~lv^if'*' 
i^^4:-.eMe•i  Njt  .l-TiMons 

hii.*<'d  •’»«  MStilfc  «>l 
rm>t»kin'Mrf\  MV  svsbii--'\«. 

Mo  M''  ii«r’  gr»Mjj 

pKibMn  -s'ivlA^ 
que*.  ntiUe.  f'l-.vlu’gs 
ar*  H'unhlf*'.' 

' 

• V(^nntea(M-v> 

VNViKvi'  w»»rk  i-.  vts»n 
nnuiiUed  rindeu^s  to 
undfivlt+nd 

Thi  organ  ./4t».>n  ►jIs* 
»*T*i'ls  ^;ihk),  hut  r*OUU 
p.irf*.  vrtnrt  fiu*-  ol 

il'«  ka»' 

r*v  rtpii»i  V’  .ltst»y;.Mii^»«.'i 
♦.  V ttse  ev  d\a ! d pfe 

t enrs  UJttkrSLUidMV  ut 

IH'hnitfiin  n»rTn^ 

Alt  nrv  h*nrs 

SturiP  *1  trtx.  .ifo  M tV\l 
wilhiMit  i1r»*M'»iotv 

Mcjiu  ti  »ini  d*r  ijteil  b-il 
IU'»  iJl'i  IR*  t 

Tt-rm**  -M*'  used  wjPimuI 
rtrlirHuf*'  k'  ^'»r  »**tml 

wr.JrtnlcivdfTV;  i' 
«iiltil^  lol 


Figure  12.12  Rubric  for  Engine  Design  Project  (Part  3)  (McRel  Institute  as  cited  in  Huba  & 
Freed,  2000,  p.  191) 


Cntcnj 


t XLifIbrrvt  I A' 
4 piMnU 


GiK>d  (B1  rtnr>it  (C,  D>  trurcrpt  (M 

^ points  2 piUiitii  I pui<>( 


tnifgrrfhs'f  . *t 
>\  't>  I*-' 


Ctaiftfritir 


Lvl'  \i 


TtiF  t^nrr 

wrilii'pt  'tj,  !•’  that 

^Hir>  lhii»U|»hoot  IV* 

'••‘•V  T»*VT*’ IV  > 

tMM>  ttiiti  ihi*  ui- 

kv*4i cv1  muitkp.t  juthivs 
Tilt  w\ir^  ha^hr\T 

4nil  |>riX)4uviJ  h\  vs  i-i  s 

IHH*  In  tirt 


In^ofOVMion  ■>  ipprnptv 
jtdy  pLkX’d  Ml  eittier  chp 
oiaji  IcktiM  jpiHntlix 
Ap^'T'rOikVH  jr\-  Umiu* 
fiMrtiitHi  .u^d  rvle4t.K\«i 


Vin*n*  b>  <»nif  *'dh».atiivi 
muJuplc  3vMV»rv  . 
.lifIviMm  fsMitt. 
pap»>r,  wtc.i- 


Ihi-rt  Alea!V^v  srti  «n^ 
5ni* imoi'n 


itdui  lUiiliDT  ippiopn 
.»Vl'  pl.vv'd  »r  eiUiiT  iHt 
nuiH  U*sl  nr  driJ^Yi'MLlik. 

.H 

*=-IV'U  1#^  Ml  t<*\*  air'.Mtm*- 
iiwompkfec' 


rhm'N  aniplr  indite U<-^ 
n^uh’pV  •iirtni'cv  iv  g 
diftf >vni  kint-  .-Jifk'tfni 

p.ip*r.  w I 


TVi*'n.»  s*.  m<Hv  rlwn  »»«t* 

VtU'l  |.«fl  cvi^v 


n“/ir-  >i  Mirrr  Jiiisplivt* 

i'»  i<n.  .upAi'on  in  ilit* 
W\*  • ^ fV-  ^pp»*tv^< 
Ap|Jtt»Jii.r-i  4tv  h«Jt«lv 
Jtxuuu'ii.wi  rvlf*r 
. . ..  id  Ml  t'-'  t 


55epv'’l  'sck'arl,  v nri 
•»»  n»l*l|»pt'  'ttk'H  4 \*  ■ til 
ddit-;t'nt  m’ltl)  >;  -t'-  'i-N. 
crurgirr\  rnntfi  KmK. 
t^nit  r iprr  t%  vn-- 

ll»m*  sii%‘  n'i«« 

^pcUrvI 

Mid«i7itir\^  thdi  time  m.*- 
iMit  Ul'O*  Ui  «fvlJ-cKvK 
jriil 

S.  tU'.-i%1**rAlV  <irnt;'|l*t  :M 
tM  llc*fWll  b 

ApiVliduiS  rttv  IK'4 
lint  ar 

n*t»^vnvriJ  III  text. 


Figure  12.13  Rubric  for  Engine  Design  Project  (Part  4)  (McRel  Institute  as  cited  in  Huba 
& Freed,  2000,  p.  191) 
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Oru» 


rv^i^n  prc%cnt-Jtic>n  iu 

rhp  pn*\i»nt«iHof\ 

Tly* 

thUn'^ui^  dl>d 

1*  irtcfrstmg  bui  viimt* 

Kis  sonut  mii'rv5tii)>;  (.'■jint'- 

TS  httni  to  h'lUm  and 

point'*  art*  •uHit'dr  the 

but  iiifhrifUnih»lk>M. 

j'A'iti  ui)5»ni/wi  " 

Ottal  t'liJa  II 

inlriOia  ii<m  jnd,-  i«i 
cnTKli»-'4*n  lift?  v*'**'it 

I iliur  Iho  cntr  Ov(  ii«»' « ■ 

(V4vk>Kit^  iv 

.U^f^rv  ti>  be  I't/  thr  i utl 
thrrr  iH»i»»mftiidui  iH»n 
cr  roni 

VL'udJ  aiJk 

Vi'siidi  oid.'^  arc  a*xd 

ViMuit  ,Uils  drv  ^txxj.  but  <t 

Mtifci  vImuiI  id* 

rhofe  div  tot*  fc*W  visual 

irCi^uuiUy  Tht-y  dv 

r<>  jvsv]  rliul 
jnd  tlvy 
sictnaf  quality' 

f.nv  AVV  'ir  ilittK  ult 

IU  iraJ 

^ppv  4nd  ll<*Tv^  (O  T4».*d 

AM}"*  .ind  r**\l  *ii 

vO'^W«*^lv  prv^m) 

IjlTIRth 

l>»»*  pf4*irtit4!ioti  L^  wttlnn 

rhv  prps»*ot,»iH.Tn  i>  Ini 

Ihf*  pitni’lilJU'.'li  j-*  Uv 

Ih*'  pfr-ATi'^ '-‘n  o,(.m 

the  i^sifincd  nn»c  limit* 

••hori  or  itH*  Imi^br 

iiM'iun  t «»■  - 

d'-irior^’-’  I'-n^  hy  »»'  i* 
rviAutCs  t.\r 

.1  ft^  I'J  KkJ  lr*»l>i  ll\  It’ll 
iM'i'iilr**  I'l  !«*.••> 

P nginrcnnp  .imlv-jv  tr. 

.i»»iih-‘-i'-  i*> 

Pn^i  iv*cTlr*y.  v*.i|- 

SJm  rM*|r/'*  ’<l'-  ; i'-  l•‘->•- 

I'rtst*!!ti\l  »vuh  '»uJficiev>l 
to  h«*  andcc*UH<il, 
but  not  n*  (Ix^i  It  lOMilts 

.tti* 

poilllv  PNriilllH-i  IT  w 

ik'uUttl  ll>4l  thk  .nuttciii’ 
l.tU' 

MHbl  III  lli\  ii\  .ulluMlK  ■ 
•ifiv'i  isp^ir'v  i'xpl.jnwxl 

1*  pr.*w*|*fist 

iVjni.iq'iT 


Figure  12.14  Rubric  for  Engine  Design  Project  (Part  5)  (McRel  Institute  as  cited  in  Huba  & 
Freed,  2000,  p.  191) 
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The  Portfolio  Assessment  Chart  below  reveals  that  all  professors  except  one  completed  all 
aspects  of  the  program  and  research  semester  successfully.  That  means  that  each  of  those  seven 
professors  completed  the  faculty  development  program  of  learning  with  significant  gain  in 
learning.  Seven  professors  completed  all  teaching  and  learning  products  during  the  faculty 
development  program;  and  all  seven  professors  fully  participated  in  the  research  semester, 
executing  experimental  research  in  the  classroom  with  their  students.  All  seven  professors 
prepared  a research  manuscript  and  submitted  it  for  publication.  However,  one  of  the  seven 
professors  did  not  complete  some  activities  as  planned;  he/she  did  not  diagnostically  analyze  the 
final  examination  and  did  not  use  the  second  and  third  performance  assessments  as  planned.  That 
individual  did  complete  the  research  as  planned  but  did  not  implement  the  full  range  of  changes 
prepared  and  planned  for  the  2006  course.  This  culminating  assessment,  college  portfolio, 
provides  evidence  that  the  program  was  very  successful,  resulting  in  significant  change  and  a 
new  range  of  teaching  and  learning  activities  for  each  professor.  The  portfolio  also  reflects  each 
professor’s  preparation  for  the  research  semester,  itemizing  the  products  developed  and  used 
during  the  2006  experimental  research  semester.  Generally,  the  portfolio  chart  reveals  the  results 
of  the  faculty  development  program  and  research  semester  and  documents  the  professors’ 
learning  and  progress  toward  new  teaching  and  learning  strategies,  as  well  as  that  toward  the 
Scholarship  of  Teaching. 
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Table  B.O.TTeaching  Portfolio  Assessment  Chart,  January  28,  2007-  CITL  Faeulty  Development 
Program ^ 


Portfolio  Product  (Artifact)  Content 
(See  Sections  of  information  following  this  summary) 

RM 

RR 

*AA 

BT 

IM 

BC 

*AG 

Self  Assessment  Baseline: 

1 . Student  Questionnaires  (f05  & f06) 

1 .  Professor  completion(s)  of  Student  Questionnaire  (f05  &f06) 

2.  Professor  completions  of  Self  Competency  Questionnaire 

(Feb.06,  May06,  Dec.06) 

3.  Program  Components  Assessments  (8) 

4.  Standard  Departmental  Course  Evaluations  (f05  & f06) 

4. Student  Grades  & End  of  Semester  Grades  (f05  & f06) 

J 

J 

J 

J 

J 

J 

J 

5.  Course  Analysis: 

5al._Course  Outline,  Embedded  Gen  Ed,  Content  Priorities 
5a2._Course  Content  Analysis  by  TM,TS,  LS,  Bl,  Dale,  etc. 

5b.  _ Instr.  Design  GAPS  Analysis  on-  TM,  TS,  LS,  B.  D 
5c.  _ Instructional  GAPS  Summary 
5d.  _ ABET/TAC/NAIT  SLO  by  Bloom’s  Analysis 
5e.  _ Course  Content  Schedule 

5f._  Teaching  Models  + Cooperative  Learning  + Study  Chart  + TM  graphic 
5g._  Course  Calendar  by  TM,  TS,  LS,  B,  D 

J 

J 

J 

J 

J 

J 

J 

***Student  Learning  Styles  Inventory:***  (NOT  REQUIRED) 
Kolb  (Extra  professional  effort  on  part  of  professors) 

Felder  (Extra  professional  effort  on  part  of  professor) 

KJ 

KJ 

NA 

NA 

NA 

NA 

Multifaceted  Assessment  System: 

5h. Multifaceted  Assessment  Plan  Graphic,  showing  course  assessments 

5i. Test  and  Test  Items  by  SLO  Chart 

5j. Assessment  Analysis  by  Bloom  (Chart) 

J 

J 

J 

J 

J 

J 

J 

6.  Traditional  Objective  Tests:: 

Test  Analvsis  ("Midterm  and  Final  Exam) 
Table  of  Specifications  tnot  included) 
Test  Item  Bank  fnot  included) 

7.  New  Midterm  Exam 

7.  New  Final  Exam 

8.  Diagnostic  Write  Ups  (MT  & F) 

J 

J 

J 

J 

J-  Partial 

Midterm  Analysis 

X Nd  analysis  far 
Final  Exam 

X Diagnnstics 
No  Analysis  in  Report 

J 

J 

Performance  Assessment  & Rubrics: 

7.  3 Complex  Performance  Assessments  with  multiple  tasks 

embedded 

7.  3 Rubrics,  one  to  score  each  Performance  Assessment 

( And  to  be  used  with  students  to  establish  standards  up  front) 

8.  Diagnostic  Write  Ups  tPA  1.2.3) 

* Copies  of  Students  Rubrics  (Hardcopies  on  file) 

7.  Electronic  copies  of  tests  and  PAs  & Rubrics 

other  Assessments  of  Individual  Choice:  Yes  for  All 

J 

J 

J 

J 

>/-Paptial 

Did  not  seem  to 
ose  PA  2,3 

X 

J 

J 

9.  Student  Centered  Course  Syllabus: 

All  new  components  and  check  off  list 

J 

J 

J 

J 

J 

J 

J 

10.  Professors’  Research: 

Completed  Data  Forms  (including  data  on  MT,  F,  PA1,2,3) 

Research  Results  Reports 

J 

J 

J 

J 

Partial  ^Midtarm 
X No  Final-  Robrics 

J 

J 

J 

12. Teaching  Portfolio  Assessment  Questionnaire 

13  Teaching  Models  Self  Assessment 

14  Teaching  Styles  Self  Assessment 

15  Student  Learning  Style  Opportunities  Assessment 

16  Outcomes  Achieved  as  Planned  by  Bloom  & Dale  Assessment 

J 

J 

J 

J 

J 

J 

J 

17.  Manuscript  to  be  submitted: 

Draft 

Final  Version  to  be  submitted  to  journal  tMav.  2007) 

J 

J 

J 

J 

X © Final 
Report,  not  article 

J 

J 

Legend:  J = akay  X = still  needed  © = net  due  yet 
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Table  B.0.2:  Teaching  Portfolio  Assessment  Questionnaire,  May  25,  2006  - CITL  Faculty  Development  Program 


Portfolio  Product 
(Artifact)  Content 

Description  of  my  Knowledge,  Skills, 
Abilities  on  Feb.  2,  2006 

Description  of  my  Knowledge,  Skills, 
Abilities  on  May  25*'',  2006 

Self  Assessment 
Baseline: 

Student 

1.  “Did  not  do  the  questionnaire  before.” 

l.“These  questionnaires  provided  me  with  a 
tool  to  gauge  the  difference  between  my 
perception  about  the  course  and  how  the 
students  see  the  course,  regarding  the  course 
goal,  delivery,  teaching  effectiveness, 
objectives,  etc.” 

2a.  “Syllabus  now  has  much  more  detail, 
including  SLO,  NIU  Gen  Ed,  ABET 
standards  and  assessments. 

b.  New  tests  are  more  objective  and  also 
many  different  types  of  questions. 
Performance  Tasks  have  been  added. 

c.  Became  aware  of  various  learning  and 
teaching  methods. 

d.  Cooperative  and  group  learning  will  take 
place.” 

Questionnaire 

Professor 
completion  of 
Student  Questionnaire 

Professor 
Self-  Competency 
Questionnaire 

2.  “The  syllabus  was  not  detailed  and 
objectives  were  not  spelled  out  in  detail. 
Testing  and  measurement  needed  to  be 
improved  (there  were  no  performance 
tasks,  rubrics,  etc.)  Students  and  I were 
not  aware  of  various  Teaching  and 
Learning  Methods.  There  was  not  much 
scope  of  Cooperative  and  Group 
Learning.” 

3a.  “Most  elements  were  missing.  I was  not 
convinced  of  the  value  of  many  elements, 
b.  Before,  I had  taken  much  of  the  “burden 
of  teaching  the  material  to  students.” 

3a.  “Most  elements  are  in  place.  They  are 
valuable.  I expect  students  to  recognize 
them. 

b.  I’m  shifting  the  burden  dramatically.” 

4.  “I  was  in  very  bad  shape.” 

4.  “I  hope,  and  I think  so;  I will  improve.” 

5.  No  comment 

5.  No  comment 

6.  “Good  knowledge  from  student 
evaluations.  Useful  to  change  course  and 
adapt  to  student  comments.” 

6.”Better,  more  detailed  knowledge;  much 
more  of  a quantitative  approach;  easier  to 
compare  data  to  previous  courses.” 

7.  “I  was  unaware  of  teaching  models  and 
learning  style  inventories.  I also  did  not 
have  a very  comprehensive  syllabus.  This 
was  evident  by  the  responses  to  all  of  the 
questionnaires.” 

7.  I did  not  know  how  the  students  would 
respond  in  their  questionnaire  at  the  end  of 
the  next  semester.  However,  I know  that  my 
responses  to  both  the  Student  and  Self- 
competency questionnaires  were  extremely 
different.  I now  have  a level  of  confidence  in 
both  content  and  terminology  related  to 
teaching.” 
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Course  Analysis: 

GAPS 

Analysis  on- 
TM,  TS,  LS,  B,  D 

ABET/TAC 

/NAIT  SLO  by 
Bloom’s  Analysis 

Course 

Calendar  by  TM, 
TS,  LS,  B,  D 

Teaching 

Models  + 
Cooperative 
Learning  + 
Mapping  Study 
Chart 


1.  “Was  not  familiar  with  these 
issues.” 


2a.  “For  GAPS  analysis,  most  of  the  terminologies 
were  not  very  clear  and  consequently  answers 
sometimes  were  more  of  a guess.” 
h.  ABET  SLO  hy  Bloom’s  analysis  reflected  more  of 
lower  level  of  learning. 

c.  Course  calendar  had  only  topics  listed  hy  weeks 
hut  not  TM,  TS,  etc.  Also,  again,  many  of  these 
terms  were  either  not  very  clear  or  not  thought  of  in 
terms  of  their  implementation  in  the  context  of  my 
course.” 

c.  There  was  not  chart  for  Teaching  Models  + 
Cooperative  Learning,  etc.” 

3.”In  other  courses,  I had  used  many  different 
learning  and  teaching  styles,  and  many  more 
assessments.  However,  I have  heen  putting  off  the 
task  of  doing  this  with  [this  course], 
h.  Was  familiar  with  Bloom  & Dale;  was  familiar 
w/Felder’s  LS  Taxonomy. 

c.  I had  never  created  such  a detailed  calendar; 

d.  Connections  to  ABET  were  weak.” 

4a.  “I  had  a little  knowledge  on  this  matter.  I knew  I 
had  to  connect  my  course  objectives  with  ABET 
outcome,  but  I was  not  clear  how  to  do  this.  I had  no 
idea  about  teaching  models.” 


5a.  “Did  not  know  much  about  this; 

b.  Knew  about  the  ABET  outcomes  and  associated 

details,  but  did  not  know  about  Bloom’s  Analysis; 

e.  Did  not  have  a course  calendar  before; 

d.  Did  not  think  much  about  TM,  CL,  and  mapping.” 


6.  “No  knowledge  previously,  but  many  components 
of  techniques  and  models  already  utilized  in 
classroom.” 


1.  “Realized  the  importance  of  GAPS  analysis, 
SLOs,  teaching  models,. ..etc.  and  how  they  can  be 
used  to  enhance  the  learning  process.” 

2.  “Course  analysis  is  more  detailed.  The 
terminology  is  clearer  and  hopefully  future  GAPS 
analyses  would  reflect  that. 

b.  ABET  SLO  by  Bloom’s  Analysis  not  only  are 
more  detailed,  but  they  reflect  higher  level  of 
learning. 

c.  Course  calendar  has  more  details  to  include 
TM,TS,  etc. 

d.  Teaching  Models  + Cooperative  Learning,  etc. 
chart  makes  us  not  only  realize  what  these  models 
mean,  but  also  their  strengths  and  weaknesses  and 
whether  they  are  applicable  for  my  course.” 

3.  “Class  formalized  much  knowledge  in  these 
areas.  Reading  gave  labels  to  much  of  what  I was 
doing,  provided  more  of  a scientific  foundation. 

b.  I know  exactly  where  I can  go  (or  begin  going) 
to  look  stuff  up,  foundations  mostly. 

c.  Much  stronger  connections  to  specific  learning 
outcomes.” 


4a.  “I  feel  really  strong  in  doing  this  connection  - 
my  course  objectives  & ABET  outcomes. 

b.  Now  I know  different  teaching  models. 

c.  I feel  I have  gained  new  tools  to  research.” 

Sa.  ’Did  Gap  Analysis,  but  need  to  revisit  the 
item;  make  it-  proper  use; 

b.  Linking  ABET  outcomes  and  SLOs  with 
Bloom’s  is  a good  exercise.  It  helps  to  plan  for 
level  of  teaching  for  each  of  these  SLOs  and  their 
connection  with  ABET  Outcomes.  This  will  also 
help  to  develop  ABET  documentations. 

c.  Course  calendar  will  provide  some  guideline 
towards  course  delivery.  However,  there  may  be 
some  time  when  it  will  be  difficulty  to  follow  it 
exactly. 

d.  Understanding  of  teaching  models  and  their 
links  with  various  learning  styles,  along  with  the 
cooperative  learning  is  verv  helpful” 

6.”Good  understanding  of  topics,  actual 
application  to  current  course.  Studied  a variety  of 
models  from  different  sources.” 


7.  “As  evidenced  by  my  courses  analyses,  I was 
lacking  in  the  implementation  of  learning  styles, 
teaching  models,  and  teaching  styles,  as  well  as 
having  a course  calendar.  There  was  much 
opportunity  for  improvement.  I had  no  relationship 
at  all  between  course  activities,  assessments,  active 
learning,  and  Bloom’s  Taxonomy.  My  student 
learning  objectives  were  very  general  and  were  not 
directly  related  via  a matrix  to  ABET  outcomes.” 


7.  I have  made  a concerted  effort  to  look  at 
incorporating  different  teaching  styles  and 
models,  as  well  as  learning  styles  into  my  courses, 
and  I am  committed  to  trying  them  during  the 
next  semester.  My  student  learning  objectives  are 
now  written  with  specific  goal  s in  mind.  There  is 
also  a direct  correlation  between  them  and  ABET 
(and  GEN  ED)  outcomes.  The  greatest  impact  I 
believe  has  been  in  the  addition  of  active 
learning/Bloom’s  concepts  into  my  course 
preparation.  It  has  proven  to  be  a truly  eye- 
opening experience.  I will  definitely  continue  with 
these  principles  in  all  of  my  future  courses.” 
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Student  Centered 
Course  Syllabus: 

All  new 


components  and 
check  off  list 


1.  “My  syllabus  was  ok,  but  not  excellent.” 


2.”The  syllabus  was  not  detailed  and  objectives 
were  not  explained  in  detail.  It  just  followed 
the  format  that  is  minimum  required  by  ABET 
but  without  the  details  (e.g. , SLO).” 

3a.  “Had  never  listed  as  much  detail,  never 
listed  SLOs; 

b.  Had  never  had  fixed  grading  scale.” 


4.  “I  had  the  opportunity  to  participate  in 
syllabus  workshop  in  [last  institution],  so  I was 
aware  of  this.” 

5.  “Past  course  syllabus  was  detailed  but  did 
not  have  all  the  SLOs  written  clearly.” 

6. ”Good  knowledge  of  syllabus  construction 
with  much  pertinent  information  given.” 


7.  “My  past  syllabus  was  not  comprehensive. 
Many  components  were  lacking.  The  SLOs 
[student  learning  outcomes]  were  not  specific 
and  I did  not  have  a checklist.” 


1.  “Syllabus  is  much  better  because  it  cannot 
guide  me  and  the  students  through  the  course 
delivery.  The  syllabus  gives  the  students  a 
full  picture  of  what  is  expected  from  them  to 
achieve  the  grade  they  want.” 

2.  “Syllabus  now  has  much  more  details, 
including  SLO,  NIL  Gen  Ed,  ABET 
standards,  and  assessments.” 


3a.  “Can  use  SLO  in  syllabus  as  advanced 
organizer. 

b.  Grading  is  much  more  clearly  defined;  less 
ambiguity. 

c.  new  syllabus  is  better  for  meta-cognition. 

d.  Students  better  able  to  assess  their 
progress.” 

4.  “I  feel  the  way  I was  using  my  syllabus  in 
the  past  was  right,  but  now  I have  improved 
in  the  course  schedule  part.” 

5.  “Course  syllabus  is  now  much  improved 
with  detailed  SLOs.” 

6a.”Improved  syllabus  design  with  much 
more  tangible  and  constructive  information 
for  the  student. 

b.  Students  will  know  course  requirements 
and  grading  policy.” 

7.  “My  new  syllabus  looks  like  a new  species 
as  compared  with  the  old  one.  It  gives  the 
students  much  more  information,  especially 
in  regard  to  expectations.  The  SLOs  are 
spelled  out  and  related  to  ABET.  I have 
incorporated  many  new  components, 
including  the  checklist.  There  has  been  an 
enormous  leap  forward.  The  final  product  is 
very  different,  but  more  importantly  I now 
have  an  understanding  of  the  components 
behind  the  syllabus.” 
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Multifaceted 

1.  “Was  not  familiar  with  charts,  test  [analysis  & 

la.  “Charts  will  help  students  connecting  the 

Assessment 

development],  performance  [tasks]  and  ruhrics.” 

different  learning  activities  to  be  able  to  visualize 

System: 

Course 

the  full  picture  of  the  learning  abilities.” 
b.  “Tests  became  a more  accurate  tool  of 
assessment  that  can  be  used  to  gauge  students’ 
performance,  since  the  tests  can  be  related  to 

Assessment  Plan 

SLOs.” 

Chart  showing 

c.  “Performance  assessments  provide  alternative 

course 

means  to  assess  the  students’  performance  and 

assessments 

ensure  fair  grades  in  the  course.” 

Assessment 

2a.  “No  multifaceted  chart  and  Assessment  analysis 

2a.  “No  only  these  charts  were  prepared,  but  the 

Analysis  by 
Bloom  (Chart) 

hy  Bloom  was  prepared.” 

chart  showed  how  many  different  ways  a student 

h.  Tests  were  not  ohjective.  There  was  not  table  of 

can  be  assessed. 

specifications.  Exam  questions  were  only  of  one  type. 

b.  New  exams  are  more  objective  and  have  various 

c.  There  were  no  performance  assessments.” 

types  of  responses.  They  also  are  designed  to  test 

various  levels  of  learning. 

c.  Three  performance  assessments  and  rubrics 

Traditional 

were  made.  These  performance  tasks 

Objective  Tests: 

consequently  involve  much  higher  level  of 
learning  and  also  certainly  involve  active  learning 

Test 

(instead  of  mostly  passive).” 

Analysis 

3a.  “Gave  mostly  long  problems,  subjectively  scored 

3a.  “Much  better  Balance  —problem  types. 

Table  of 

with  five  quizzes  before.” 

conceptual  vs.  putting  all  pieces  together  vs. 

b.  My  number  of  major  assessments  was  relatively 

creating  something. 

Specifications 

high.  However,  I did  not  have  performance  items  in 

b.  Well  defined  rubrics  for  subjective  problem 

the  same  way  that  I have  now.  Not  as  deep  (Blooms). 

tests. 

Test  Item 

c.  Assessment  problems  were  not  necessarily 

c.  Now  they  are 

Bank 

distributed  well  over  the  learning  objective; 

d.  Now  I do.” 

d.  I had  a few  multiple  choice  items,  but  not  enough 

e.f.g.  Now  I do. 

New 

to  form  a broad  assessment  of  understanding.” 

h.  Will  be  stronger;  will  be  assessed;  will  require 

Midterm  Exam 

e.  Never  had  a high  level  performance  in  [this  course] 
before. 

students  to  prepare.” 

New  Final 

f.  Never  had  detailed  rubrics  for  scoring  reports. 

g.  Never  thought  of  having  students  score  themselves 

Exam 

w/rubric. 

h.  In  class  group  work  in  big  lecture  was  weak.” 

Test  Items 

4a.”Definitely,  I know  much  more  now.  I really 

by  SLO 

think  that  the  assessment  map  is  a very  good  tool 

Chart 

4a.  “I  barely  knew  anything  about  Bloom’s 

that  will  allow  me  to  assess  myself  - where  I am. 

Taxonomy.” 

b.  I feel  I know,  and  understand  now,  that  there 

b.  My  knowledge  on  this  matter  was  really  poor. 

are  multiple  ways  to  assess  students. 

Performance 
Assessment  & 

c.  I didn’t  know  anything  about  this.” 

c.  I know  how  to  construct  a performance  task 
and  rubric.  I think  this  is  just  the  beginning,  but 
at  least  I know  where  to  start  from.” 

Rubrics: 

ja.  “Assessment  chart  is  quite  helpful  to  see  the 

3 Complex 

total  assessment  process  for  the  course; 

Performance 

5a.  “Did  not  have  an  assessment  chart  before; 

b.  Assessment  analysis  by  Bloom  shows  the  level  of 

Assessments  with 

b.  Did  not  have  any  assessment  analysis  by  Bloom’s; 

performance  that  students  are  going  to  perform 

multiple  tasks 

c-  Did  not  have  any  test  analysis  before; 

for  each  of  the  assessment  methods. 

embedded 

d.  Did  not  have  any  table  of  specification; 

e.  There  was  no  test  item  bank; 

e.  Test  analysis  provides  in-depth  understanding 

3 Rubrics, 

f.  Did  not  use  many  objective  tests  for  midterm  or 

while  designing  and  improving  a test; 

Inals. 

d.  Item  bank  is  very  helpful  from  where  one  can 

g.  Have  used  performance  test  before  [but  not 

pick  required  tests  items  at  desired  level  of  Bloom; 

Assessment  (and 

formally  written  as  performance  tasks]; 

e.  In  addition  to  developed  midterm  and  final,  I 

h.  Used  rubrics  in  limited  way.” 

may  need  to  add  some  subjective  test  items. 

to  be  used  with 

f.  Developed  performance  tests  provide  broad 

students  to  esta- 

range  of  assessment; 

Wish  standards  up 

g.  Rubrics  will  be  very  helpful  both  for  the 

front) 

students  and  for  the  faculty.” 
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6a.”Limited  to  very  low  knowledge  in  these  areas. 

b.  Very  good  knowledge  in  test  construction,  but  no 
quantitative,  statistical  analyses  [before  now].” 

c.  Very  good  previous  knowledge  in  assessments  and 
scoring  rubrics;  Scoring  rubrics  already  extensively 
utilized  in  courses.” 


6a.” Ability  to  execute  charges  and  to  rank 
assessment  activities  by  creativity  and  active 
versus  passive  learning  processes. 

b.  Analytical  tools  useful  for  critical  and  detailed 
assessment  of  test  questions.  Valuable  test  bank 
that  connects  individual  test  questions  with 
student  learning  objectives  and  outcomes. 

c.  Able  to  expand  performance  tasks  to  higher 
Bloom’s  comprehension  levels  with  a more  active 
approach  for  the  student.  Included  more  group 
learning.  Able  to  fine  tune  rubrics  for  assessment 
performance  tasks.” 


CONTINUE: 

Multifaceted 

Assessment 

System: 


7.  “I  did  have  multiple  assessment  types  prior  to  this 
workshop.  However,  I did  not  know  exactly  what  the 
terminology  and  importance  of  each  one  was. 

My  traditional  tests  were  mainly  ‘short  answer’  or 
‘essay’.  I had  never  before  performed  a ‘test 
analysis’.  The  analysis  of  my  past  tests  did  show  that 
I was  testing  what  I wanted  to  test,  and  the  difficulty 
was  where  I wanted  it  to  be.  I do  not  want  that  to 
change  in  the  future.  I did  not  have  my  test  items 
written/grouped  by  SLOs.” 

“I  had  incorporated  complex  performance 
assessments  with  multiple  embedded  tasks. 

However,  I was  not  using  this  terminology.  I also  did 
not  use  rubric  scoring,  although  I was  aware  of  this 
technique.” 


7.  “I  created  a chart  of  the  assessments  to  be  used 
in  the  next  semester.  I have  included  more  than  I 
had  in  the  past.  Group  discussion  will  now  be 
assessed,  rather  than  just  performed.  The 
students  will  know  what  they  have  to  do  because 
of  the  rubric  scoring.” 

“I  rewrote  my  midterm  and  final  exam  to  be  much 
more  ‘objective’  in  nature,  including  questions 
that  are  multiple  choice,  true/false,  etc.  I am  not 
confident  this  will  sit  well  with  the  students.  I 
believe  that  short  answer  problem  solving  types  of 
questions  are  still  objective,  but  they  give  the 
students  the  opportunity  to  use  their  own 
approach  to  come  up  with  the  ‘correct’  answer.  I 
will  try  the  new  tests  and  see  what  happens.  I will 
definitely  use  the  test  analysis  during  the  next 
semester.  I will  use  this  in  all  of  my  classes.” 


Other 

Assessments  of 

Individual 

Choice; 


“I  created  three  new  performance  assessments 
and  corresponding  rubrics.  This  area  excites  me 
the  most  because  I can  see  where  the  students  will 
be  reaching  the  upper  levels  of  Bloom’s.  I can  also 
have  group  discussions  surrounding  the  PAs, 
where  many  more  teaching  models  can  be  used.  I 
see  this  as  a new  door  being  opened.  I am  anxious 
to  see  the  outcomes  next  semester.  I believe  that 
the  rubrics  are  key  to  the  success  of  proper 
assessment  of  performance  type  tasks.  I hope  the 
students  will  also  see  the  value  in  knowing  my 
expectations  of  them.” 


List  and  Describe 
Here: 
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Teaching  Portfolio  Assessment  Chart  Form,  DA" 

rE  - CITE  Facul 

Ity  Development  Program 

Portfolio  Product  (Artifact)  Content 
(See  Sections  of  information  following  this  summary) 

Faculty 

Member 

Faculty 

Member 

Self  Assessment  Baseline: 

1 . Student  Questionnaires  (f05  & f06) 

1 .  Professor  completion(s)  of  Student  Questionnaire  (f05  &f06) 

2.  Professor  completions  of  Self  Competency  Questionnaire 

(Feb.06,  May06,  Dec.06) 

3.  Program  Components  Assessments  (8) 

4.  Standard  Departmental  Course  Evaluations  (f05  & f06) 

4. Student  Grades  & End  of  Semester  Grades  (f05  & f06) 

5.  Course  Analysis: 

5al._Course  Outline,  Embedded  Gen  Ed,  Content  Priorities 
5a2._Course  Content  Analysis  by  TM,TS,  LS,  Bl,  Dale,  etc. 

5b.  _ Instr.  Design  GAPS  Analysis  on-  TM,  TS,  LS,  B,  D 
5c.  _ Instructional  GAPS  Summary 
5d.  _ ABET/TAC/NAIT  SLO  by  Bloom’s  Analysis 
5e.  _ Course  Content  Schedule 

5f._  Teaching  Models+Cooperative  Learning+Study  Chart+TM  graphic 
5g._  Course  Calendar  by  TM,  TS,  LS,  B,  D 

***Student  Learning  Styles  Inventory:***  (NOT  REQUIRED) 
Kolb  (Extra  professional  effort  on  part  of  professors) 

Felder  (Extra  professional  effort  on  part  of  professor) 

Multifaceted  Assessment  System: 

5h. Multifacted  Assessment  Plan  Graphic,  showing  course  assessments 

5i. Test  and  Test  Items  by  SLO  Chart 

5j. Assessment  Analysis  by  Bloom  (Chart) 

6.  Traditional  Objective  Tests:: 

Test  Analysis  (Midterm  and  Final  Exam) 
Table  of  Specifications  (not  included) 
Test  Item  Bank  (not  included) 

7.  New  Midterm  Exam 

7.  New  Final  Exam 

8.  Diagnostic  Write  Ups  (MT  & F) 

Performance  Assessment  & Rubrics: 

7.  3 Complex  Performance  Assessments  with  multiple  tasks 

embedded 

7.  3 Rubrics,  one  to  score  each  Performance  Assessment 

( And  to  be  used  with  students  to  establish  standards  up  front) 

8.  Diagnostic  Write  Ups  (PA  1.2.3) 

* Copies  of  Students  Rubrics  (Hardcopies  on  file) 

7.  Electronic  copies  of  tests  and  PAs  & Rubrics 

other  Assessments  of  Individual  Choice:  Yes  for  All 

9.  Student  Centered  Course  Syllabus: 

All  new  components  and  check  off  list 

10.  Professors’  Research: 

Completed  Data  Forms  (including  data  on  MT,  F,  PA1,2,3) 

Research  Results  Reports 

12. Teaching  Portfolio  Assessment  Questionnaire 

13  Teaching  Models  Self  Assessment 

14  Teaching  Styles  Self  Assessment 

15  Student  Learning  Style  Opportunities  Assessment 

16  Outcomes  Achieved  as  Planned  by  Bloom  & Dale  Assessment 

17.  Manuscript  to  be  submitted: 

Draft 

Final  Version  to  be  submitted  to  journal  fMav.  2007) 

Legend:  J = akay  X = still  needed  © = net  due  yet  See  Result 
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PROFESSORS’  SELF  PERCEPTIONS  OF  COMPETENCY 
Jule  Scarborough,  Ph.D.  and  Jerry  Gilmer,  Ph.D. 

A twenty-five  item  survey  was  developed  to  determine  the  professors’  self  assessment  of  their  competency  levels  in  the 
main  areas  covered  during  this  project;  Analysis,  Assessment,  Methods,  and  Research  (Scarborough,  2006).  The  survey 
was  administered  three  times  during  the  project;  immediately  prior  to  the  beginning  of  the  formal  teaching  sessions, 
immediately  after  the  conclusion  of  the  formal  teaching  sessions,  and  at  the  end  of  the  semester  during  which  the  professors 
where  practicing  the  concepts  learned.  Each  item  was  worth  4 points  for  a maximum  total  of  100  points.  The  table  below 
contains  the  self  competency  scores  for  the  professors  for  each  content  area  and  each  administration  of  the  survey.*  See 
Questionnaire  in  B.l.a  and  C.2 


Table  B. 1.1 


Content  Area: 
Max.  Score: 


Analysis 

36 


Assessment  Methods 


Research 


32 


24 


8 


Total 

100 


Admin  1 
(2/2/06) 


Abdel-Motaleb 

18 

50%* 

11 

34% 

10 

42% 

3 

38% 

42 

42% 

Azad 

26 

72% 

22 

69% 

12 

50% 

5 

63% 

65 

65% 

Coller 

21 

58% 

17 

53% 

14 

58% 

4 

50% 

56 

56% 

Gupta 

19 

53% 

20 

63% 

11 

46% 

4 

50% 

54 

54% 

Moraga 

12 

33% 

13 

41% 

6 

25% 

3 

38% 

34 

34% 

Rahn 

21 

58% 

23 

72% 

11 

46% 

4 

50% 

59 

59% 

Tatara 

23 

64% 

18 

56% 

16 

67% 

6 

75% 

63 

63% 

Admin  1 mean 

20 

20% 

18 

18% 

11 

11% 

4.1 

4% 

53 

53% 

Admin  2 

(6/16/06) 

Abdel-Motaleb 

24 

67% 

21 

66% 

18 

75% 

4 

50% 

67 

67% 

Azad 

32 

89% 

27 

84% 

19 

79% 

7 

88% 

85 

85% 

Coller 

30 

83% 

30 

94% 

21 

88% 

7 

88% 

88 

88% 

Gupta 

32 

89% 

27 

84% 

18 

75% 

6 

75% 

83 

83% 

Moraga 

29 

81% 

24 

75% 

16 

67% 

6 

75% 

75 

75% 

Rahn 

33 

92% 

32 

100% 

20 

83% 

6 

75% 

91 

91% 

Tatara 

33 

92% 

31 

97% 

21 

88% 

7 

88% 

92 

92% 

Admin  2 Mean 

30 

30% 

27 

27% 

19 

19% 

6.1 

6% 

83 

83% 

Admin  3 

(12/15/06) 

Abdel-Motaleb 

25 

69% 

25 

78% 

21 

88% 

7 

88% 

78 

78% 

Azad 

30 

83% 

27 

84% 

17 

71% 

7 

88% 

81 

81% 

Coller 

31 

86% 

29 

91% 

21 

88% 

7 

88% 

88 

88% 

Gupta 

29 

81% 

30 

94% 

17 

71% 

6 

75% 

82 

82% 

Moraga 

30 

83% 

29 

91% 

17 

71% 

7 

88% 

83 

83% 

Rahn 

34 

94% 

32 

100% 

18 

75% 

7 

88% 

91 

91% 

Tatara 

28 

78% 

26 

81% 

18 

75% 

6 

75% 

78 

78% 

Admin  3 Mean 

30 

30% 

28 

28% 

18 

18% 

6.7 

7% 

83 

83% 

* Interpretation;  A score  of  18  is  50%  of  the  maximum  possible  score  of  36  for  Analysis. 


* The  items  from  the  survey  contributing  to  the  score  for  each  content  area  are  Analysis  - 1,  2,  3,  7,  9,  16,  20,  23,  25; 
Assessment  - 4,  5,  6,  8,  10,  11,  12,  13;  Methods  - 14,  15,  17,  18,  19,  21;  Research  - 22,  24. 
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The  differences  between  the  content  area  means  across  the  different  administration  times  were  analyzed  statistically  and  the 
results  are  presented  in  the  table  below.  The  significance  levels  are  from  paired  samples  t-tests  on  the  data  from  seven 
professors  (df  = 6). 


Table  B.  1.2 


Difference:  Difference: 


Means 

Admin  2 

- Admin  1 

Admin  3 

- Admin  1 

Index 

Admin  1 
(2/2/06) 

Admin  2 
(6/16/06) 

Admin  3 
(12/15/06) 

Difference 

Sig. 

Level 

Difference 

Sig. 

Level 

Analysis  (36)** 

20.0 

30.4 

29.6 

10.4 

0.000 

9.6 

0.002 

Assessment  (32) 

17.7 

27.4 

28.3 

9.7 

0.000 

10.6 

0.000 

Methods  (24) 

11.4 

19.0 

18.4 

7.6 

0.000 

7.0 

0.001 

Research  (8) 

4.1 

6.1 

6.7 

2.0 

0.001 

2.6 

0.003 

Overall  (100) 

53.3 

83.0 

83.0 

29.7 

0.000 

29.7 

0.001 

**  - The  number  in  parentheses  is  the  maximum  score  for  the  index. 


The  data  in  the  table  indicate  that  the  professors’  increase  in  self  competence  from  the  first 
administration  to  the  second  was  statistically  significant  and  that  the  increase  remained  strong, 
and  statistically  significant,  several  months  later  after  the  professors  were  able  to  practice  the 
concepts  in  their  own  classrooms. 
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Point  value  for  each  item  is  from  1 to  4 - the  same  as  the  response  labels.  Maximum  possible  points  = 100. 


COMPETENCY  SELF  ASSESSMENT 
CEET  INITIATIVE  ON  TEACHING  AND  LEARNING 
Jule  Dee  Scarborough  (2006) 


Please  respond  to  eaeh  question  about  the  level  of  knowledge,  skill,  and  eonfidenee  you  feel  you  have: 


1.  Design  and  develop  courses  where  student  learning  objectives  and  outcomes  are  clear  and  distinctly  different. 


1 2 3 

Lack  sufficient  Possess  some  knowledge  and  Possess  some  knowledge 

knowledge,  skills,  skills,  but  lack  sufficient  and  skills;  feel  somewhat 

confidence  to  apply  confidence  to  apply  confident  to  apply 


4 

Possess  high  level 
knowledge  and  skills; 
confident  to  apply 


2.  Designing  and  developing  a logical  and  organized  course  where  course  syllabi  are  structured  such  that  students 
fully  understand  what  is  to  happen  for  the  entire  semester;  where  the  syllabus  is  the  course  map  for  both  myself 
(professor)  and  the  students;  and  where  the  syllabus  provides  all  information  about  objectives,  course  content, 
timeline,  course  requirements,  student  learning  assessments  (tests,  projects,  etc.)  grading  structure  and  criteria,  etc. 

12  3 4 

Lack  sufficient  Possess  some  knowledge  and  Possess  some  knowledge  Possess  high  level 

knowledge,  skills,  skills,  but  lack  sufficient  and  skills;  feel  somewhat  knowledge  and  skills; 

confidence  to  apply  confidence  to  apply  confident  to  apply  confident  to  apply 

3.  Providing  learning  activities  that  align  with  the  syllabus  and  course  content  identified  in  the  syllabus;  adhering  to 
the  timeline  in  the  syllabus;  and  leading  student  learning  without  significant  distractions  or  deviations  unrelated  to 
content  where  each  lesson  and  learning  activity  are  directly  related  and  add  value;  where  no  unplanned,  last 
minute,  or  major  assignments  not  identified  on  the  syllabus  are  imposed  upon  students  unexpectedly. 

12  3 4 

Lack  sufficient  Possess  some  knowledge  and  Possess  some  knowledge  Possess  high  level 

knowledge,  skills,  skills,  but  lack  sufficient  and  skills ; feel  somewhat  knowledge  and  skills; 

confidence  to  apply  confidence  to  apply  confident  to  apply  confident  to  apply 

4.  Designing  a total  student  assessment  system  where  there  is  a great  variety  of  types  of  student  learning  assessment, 
tests,  quizzes,  case  studies  in  industry,  literature  studies,  research,  papers  or  other  writing  assignments,  projects, 
presentations,  portfolios,  etc. 

12  3 4 

Lack  sufficient  Possess  some  knowledge  and  Possess  some  knowledge  Possess  high  level 

knowledge,  skills,  skills,  but  lack  sufficient  and  skills ; feel  somewhat  knowledge  and  skills; 

confidence  to  apply  confidence  to  apply  confident  to  apply  confident  to  apply 


5.  Designing  student  assessments  that  directly  align  and  measure  knowledge  and/or  skills  itemized  on  course  syllabi. 


1 

Lack  sufficient 
knowledge,  skills, 
confidence  to  apply 


2 

Possess  some  knowledge  and 
skills,  but  lack  sufficient 
confidence  to  apply 


3 

Possess  some  knowledge 
and  skills;  feel  somewhat 
confident  to  apply 


4 

Possess  high  level 
knowledge  and  skills; 
confident  to  apply 
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6.  Implementing  a student  learning  assessment  strategy  throughout  the  course  where  feedback  on  all  student 
assignments,  or  learning  assessments  is  immediate  (or  reasonably  timed  - e.g.,  2 weeks);  in  other  words,  students 
receive  feedback  from  the  professor  on  grades  or  scores  for  tests,  projects,  etc.  that  can  be  considered  immediate  in 
the  university  schedule  context  (e.g.,  1-3  classes  later). 

12  3 4 

Lack  sufficient  Possess  some  knowledge  and  Possess  some  knowledge  Possess  high  level 

knowledge,  skills,  skills,  but  lack  sufficient  and  skills ; feel  somewhat  knowledge  and  skills; 

confidence  to  apply  confidence  to  apply  confident  to  apply  confident  to  apply 

7.  Designing  and  developing  instruction  using  Bloom’s  Taxonomy  of  levels  of  learning  where  the  upper  levels  of 
Bloom’s  are  achieved  many  times  throughout  the  course  (e.g.,  levels-knowledge,  comprehension,  application, 
synthesis,  analysis,  and  evaluation). 

12  3 4 

Lack  sufficient  Possess  some  knowledge  and  Possess  some  knowledge  Possess  high  level 

knowledge,  skills,  skills,  but  lack  sufficient  and  skills;  feel  somewhat  knowledge  and  skills; 

confidence  to  apply  confidence  to  apply  confident  to  apply  confident  to  apply 

8.  Designing  and  developing  student  learning  assessments  (e.g.,  tests,  performances)  using  Bloom’s  Taxonomy  of 
levels  of  learning  where  the  upper  levels  of  Bloom’s  are  achieved  many  times  throughout  the  course  (e.g.,  levels- 
knowledge,  comprehension,  application,  synthesis,  analysis,  and  evaluation). 


1 2 3 

Lack  sufficient  Possess  some  knowledge  and  Possess  some  knowledge 

knowledge,  skills,  skills,  but  lack  sufficient  and  skills ; feel  somewhat 

confidence  to  apply  confidence  to  apply  confident  to  apply 

9.  Developing  a course  using  the  “reversed  design”  process. 

1 2 3 

Lack  sufficient  Possess  some  knowledge  and  Possess  some  knowledge 

knowledge,  skills,  skills,  but  lack  sufficient  and  skills ; feel  somewhat 

confidence  to  apply  confidence  to  apply  confident  to  apply 


4 

Possess  high  level 
knowledge  and  skills; 
confident  to  apply 


4 

Possess  high  level 
knowledge  and  skills; 
confident  to  apply 


10.  Designing  and  developing  traditional  tests  that  directly  measure  what  students  are  responsible  for  learning  from 
professors  and  other  sources  or  activities  identified  in  the  syllabi;  where  the  items  are  an  adequate  sample  of  the 
content  domain  and  the  item  types  are  appropriate  to  the  purposes  of  the  test;  where  the  items  require  thinking  at 
the  upper  levels  of  Bloom’s  Taxonomy;  where  the  scoring  and  grading  procedures  are  transparent  to  the  student 
and  as  objective  as  possible;  where  each  item  can  be  traced  directly  back  to  the  standards,  student  learning 
objectives,  text  and  source  information,  and  learning  experiences. 


1 

Lack  sufficient 
knowledge,  skills, 
confidence  to  apply 


2 

Possess  some  knowledge  and 
skills,  but  lack  sufficient 
confidence  to  apply 


3 

Possess  some  knowledge 
and  skills;  feel  somewhat 
confident  to  apply 


4 

Possess  high  level 
knowledge  and  skills; 
confident  to  apply 


11.  Performing  test  analysis  involving  statistical  analysis  of  students’  responses;  obtaining  item  statistics  for 
judging  and  improving  the  quality  of  individual  items  and  for  judging  and  improving  the  quality  of  instruction. 


1 

Lack  sufficient 
knowledge,  skills, 
confidence  to  apply 


2 

Possess  some  knowledge  and 
skills,  but  lack  sufficient 
confidence  to  apply 


3 

Possess  some  knowledge 
and  skills;  feel  somewhat 
confident  to  apply 


4 

Possess  high  level 
knowledge  and  skills; 
confident  to  apply 
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12.  Developing  student  performance  tasks  to  measure  what  students  can  do  with  knowledge;  where  there  is  a direct 
link  to  traditional  tests  that  move  student  learning  assessment  from  “knowing  about”  to  “performing  or  doing-using 
knowledge”;  where  the  upper  levels  of  Bloom’s  Taxonomy  or  levels  of  learning  are  demonstrated. 

12  3 4 

Lack  sufficient  Possess  some  knowledge  and  Possess  some  knowledge  Possess  high  level 

knowledge,  skills,  skills,  but  lack  sufficient  and  skills ; feel  somewhat  knowledge  and  skills; 

confidence  to  apply  confidence  to  apply  confident  to  apply  confident  to  apply 

13.  Designing  and  developing  rubrics  for  the  purpose  of  scoring  (grading)  student  performances  that  measure 
"learning  by  doing"  where  students  demonstrate  what  they  know  by  using  knowledge  to  perform  a task,  etc. 

12  3 4 

Lack  sufficient  Possess  some  knowledge  and  Possess  some  knowledge  Possess  high  level 

knowledge,  skills,  skills,  but  lack  sufficient  and  skills;  feel  somewhat  knowledge  and  skills; 

confidence  to  apply  confidence  to  apply  confident  to  apply  confident  to  apply 

14.  Designing  and  developing  a course  where  many  different  teaching  models  are  used  throughout  the  course  (e.g., 
lecture,  direct  instruction,  reciprocal,  reciprocal-performance,  mastery  learning,  graphic  organizers,  concept 
attainment,  conceptualization,  inductive  thinking,  deductive  thinking,  concept  formation,  inquiry,  training, 
synectics,  psychomotor,  metaphorical,  non-directive,  role  play,  cooperative/collaborative,  etc.). 

12  3 4 

Lack  sufficient  Possess  some  knowledge  and  Possess  some  knowledge  Possess  high  level 

knowledge,  skills,  skills,  but  lack  sufficient  and  skills;  feel  somewhat  knowledge  and  skills; 

confidence  to  apply  confidence  to  apply  confident  to  apply  confident  to  apply 

15.  Designing  and  developing  a course  where  many  different  teaching  styles  are  used  throughout  the  semester  (e.g., 
command,  practice,  self-check,  inclusion,  guided  discovery,  convergent  discovery,  divergent  production,  learner- 
designed,  learner-initiated,  self-teaching  styles). 

12  3 4 

Lack  sufficient  Possess  some  knowledge  and  Possess  some  knowledge  Possess  high  level 

knowledge,  skills,  skills,  but  lack  sufficient  and  skills ; feel  somewhat  knowledge  and  skills; 

confidence  to  apply  confidence  to  apply  confident  to  apply  confident  to  apply 

16.  Designing  and  developing  a course  that  accommodates  different  learning  intelligences  and  student  learning 
styles  as  "characteristic  of  the  cognitive,  affective,  and  physiological  behaviors  that  serve  as  relatively  stable 
indicators  of  how  learners  perceive,  interact  with,  and  respond  to  the  learning  environment"  or  "the  way  each 
learner  begins  to  concentrate,  process,  and  retain  new  and  difficult  information"  or  "concrete  experience,  reflective 
observation,  abstract  conceptualization,  active  experimentation." 

12  3 4 

Lack  sufficient  Possess  some  knowledge  and  Possess  some  knowledge  Possess  high  level 

knowledge,  skills,  skills,  but  lack  sufficient  and  skills ; feel  somewhat  knowledge  and  skills; 

confidence  to  apply  confidence  to  apply  confident  to  apply  confident  to  apply 

17.  Designing  and  developing  a course  where  the  burden  of  learning  is  on  the  student  rather  than  the  professor; 
where  the  professor  assumes  instructional  leadership  and  directs  student  learning  but  is  not  entirely  responsible  for 
“imparting”  all  the  knowledge  to  be  learned  “directly”;  where  students  actively  engage  in  their  own  learning;  where 
lecture  is  not  KING.  BE  HONEST! 

12  3 4 

Lack  sufficient  Possess  some  knowledge  and  Possess  some  knowledge  Possess  high  level 

knowledge,  skills,  skills,  but  lack  sufficient  and  skills ; feel  somewhat  knowledge  and  skills; 

confidence  to  apply  confidence  to  apply  confident  to  apply  confident  to  apply 
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18.  Designing  and  developing  student  learning  activities  where  “cooperative”  or  “collaborative”  learning  models 
are  used  formally;  where  “informal,”  “formal,”  or  “base”  structures  are  used  based  upon  Kagan  or  Johnson  and 
Johnson  strategies  throughout  the  course  or  for  major  learning  activities. 

12  3 4 

Lack  sufficient  Possess  some  knowledge  and  Possess  some  knowledge  Possess  high  level 

knowledge,  skills,  skills,  but  lack  sufficient  and  skills ; feel  somewhat  knowledge  and  skills; 

confidence  to  apply  confidence  to  apply  confident  to  apply  confident  to  apply 

19.  Designing  and  developing  student  learning  activities  where  cooperative  learning  in  small  groups  provides  the 
opportunity  for  students  to  experience  accomplishing  a goal  together;  where  students  would  feel  that  they  are 
gaining  self-esteem  and  respect  from  others,  that  they  are  learning  more  because  they  are  learning  with  others,  that 
the  experience  raised  everyone's  learning  and  consequently  their  grades;  where  higher  level  thinking  occurs  because 
of  students  engaging  in  inquiry  together  and  asking  questions  of  each  other;  where  social  skills  develop  as  an 
outcome;  and  where  they  increase  their  capacity  to  cope  with  stress  or  adversity  - especially  where  group  learning  is 
designed  with  performance  criteria  and  where  they  were  trained  or  educated  about  group  behavior  or  dynamics. 

12  3 4 

Lack  sufficient  Possess  some  knowledge  and  Possess  some  knowledge  Possess  high  level 

knowledge,  skills,  skills,  but  lack  sufficient  and  skills;  feel  somewhat  knowledge  and  skills; 

confidence  to  apply  confidence  to  apply  confident  to  apply  confident  to  apply 

20.  Designing  and  developing  learning  activities  where  multiple  intelligences  are  required  for  learning. 

12  3 4 

Lack  sufficient  Possess  some  knowledge  and  Possess  some  knowledge  Possess  high  level 

knowledge,  skills,  skills,  but  lack  sufficient  and  skills;  feel  somewhat  knowledge  and  skills; 

confidence  to  apply  confidence  to  apply  confident  to  apply  confident  to  apply 

21.  Designing  and  developing  problem-based  learning  where  students  are  assigned  a problem  with  conditions, 
constraints  and/or  possibilities  that  require  materials  (sometimes),  research,  and/or  collaboration;  where  students 
have  to  take  responsibility  for  their  own  learning  by  solving  the  problem;  where  the  problem  crosses  the  boundaries 
of  disciplines,  entwining  theory  and  practice;  where  there  is  a focus  on  the  processes  of  knowledge  acquisition  rather 
than  the  products  of  such  processes;  where  the  professor  is  a facilitator  rather  than  instructor;  and  where  students 
will  engage  in  self  and  peer  assessment. 

12  3 4 

Lack  sufficient  Possess  some  knowledge  and  Possess  some  knowledge  Possess  high  level 

knowledge,  skills,  skills,  but  lack  sufficient  and  skills;  feel  somewhat  knowledge  and  skills; 

confidence  to  apply  confidence  to  apply  confident  to  apply  confident  to  apply 

22.  Engaging  in  the  scholarship  of  teaching,  research  in  the  classroom  on  teaching  and  student  learning  using 
appropriate  research  design  and  methodology,  analytical  or  statistical  procedures,  etc. 


1 

Lack  sufficient 
knowledge,  skills, 
confidence  to  apply 


2 

Possess  some  knowledge  and 
skills,  but  lack  sufficient 
confidence  to  apply 


3 

Possess  some  knowledge 
and  skills;  feel  somewhat 
confident  to  apply 


4 

Possess  high  level 
knowledge  and  skills; 
confident  to  apply 


23.  Using  course  evaluation  data  or  information  as  feedback  to  determine  course  changes. 


1 

Lack  sufficient 
knowledge,  skills, 
confidence  to  apply 


2 

Possess  some  knowledge  and 
skills,  but  lack  sufficient 
confidence  to  apply 


3 

Possess  some  knowledge 
and  skills;  feel  somewhat 
confident  to  apply 


4 

Possess  high  level 
knowledge  and  skills; 
confident  to  apply 
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24.  Evaluating  the  effectiveness  of  "interventions"  or  course  changes  to  improve  student  learning. 

12  3 4 

Lack  sufficient  Possess  some  knowledge  and  Possess  some  knowledge  Possess  high  level 

knowledge,  skills,  skills,  but  lack  sufficient  and  skills ; feel  somewhat  knowledge  and  skills; 

confidence  to  apply  confidence  to  apply  confident  to  apply  confident  to  apply 


25.  Closing  the  feedback  loop  and  actually  make  course  changes  for  the  purpose  of  improving  student  learning. 


1 


2 


3 


4 


Lack  sufficient 
knowledge,  skills, 
confidence  to  apply 


Possess  some  knowledge  and 
skills,  but  lack  sufficient 
confidence  to  apply 


Possess  some  knowledge 
and  skills;  feel  somewhat 
confident  to  apply 


Possess  high  level 
knowledge  and  skills; 
confident  to  apply 
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COMPARISON  EVALUATION  AND  SUMMARY 
FALL  2005  ^ FALL  2006 
(See  Portfolio  Sections  A.5) 

Jule  Dee  Scarborough,  Ph.D. 

Overall,  the  original  syllabi  for  the  baseline  courses,  Fall  2005,  were  in  need  of  categorical  additions  and 
other  improvements.  For  example,  when  reviewing  the  Summary  Chart,  there  are  very  few  schedules  of 
topics,  course  activities,  assessments,  due  dates;  there  is  also  no  mention  of  items  such  as  self-check  lists, 
general  education  goals,  course  purpose,  lab  notes,  faculty  policies  or  expectations,  professor’s  roles  and 
responsibilities,  or  that  of  the  students,  course  references,  support  services,  and/or  grading  values.  Although 
individuals  varied  in  their  approaches  and  syllabus  content,  for  the  most  part,  there  were  gaps  between  the 
model  syllabus,  which  was  an  example  that  reflected  what  the  literature  would  identify  as  a super  syllabus 
and  the  professors’  syllabi.  A super  syllabus  is  one  where  all  important  information  is  available  so  students 
can  understand:  exactly  what  they  are  going  to  learn  about  and  do,  the  assessments  and/or  requirements,  the 
course  schedule,  timelines  or  due  dates,  all  expectations,  and  more.  It  is  important  to  note  that  several  syllabi 
were  fairly  good.  But  if  defining  “good,”  “excellent,”  or  “super”  by  the  new  expectations  set  as  criteria  for 
faculty  to  achieve  in  the  program,  then  all  syllabi  needed  some  changes. 

All  the  professors  improved  their  syllabi,  some  more  dramatically  than  others,  but  there  were  major 
improvements.  However,  when  judging  them  by  the  established  criteria,  most  still  fell  short  of  being  judged 
as  “excellent”  or  “super.”  They  were  all,  however,  “good.”  GN  was  used  as  the  notation,  meaning  that 
although  they  were  good,  there  was  a need  to  add  information  professors  judged  not  necessary  to  include 
(e.g.  Support  Services  [special  accommodations  for  students  with  challenges]  or  Course  Purpose  [statement 
showing  the  course’s  relationship  to  the  field  or  real-world],  and  several  others).  It  is  important  to  note  there 
are  philosophical  or  preference  differences  among  faculty  members  as  to  what  is  deemed  important  to 
include  on  a syllabus.  Just  because  the  program  leader  provides  literature  with  criteria,  and  an  example 
reflecting  the  literature,  does  not  necessarily  convince  professors  that  they  should  include  all  recommended 
components  on  their  syllabi  or  that  they  are  necessary.  Smile. . . 

Finally,  the  professors  spent  so  much  time  analyzing  their  courses  and  developing  new  products,  they  did 
not  spend  enough  time  formatting  the  syllabi.  The  syllabus  is  critical  to  students.  And  even  though  it  may 
have  all  the  information  needed  in  it,  if  the  formatting  is  not  such  that  they  can  easily  discern  what  the 
syllabus  is  telling  them,  then  it  is  not  very  effective  and  can  actually  reduce  their  understanding  of  course 
expectations.  There  was  a need  for  all  professors  participating  in  the  program  to  reformat  their  syllabi 
visually  (spending  time  on  organization,  blocking  content,  etc.),  making  the  syllabus  possible  to  understand 
and  use  as  the  guiding  document  to  achieve  the  course  outcomes.  Some  needed  to  add  information  as  well. 

Regardless  of  the  remaining  suggested  improvements,  there  was  significant  change  and  improvement 
hy  all  professors.  This  program  component  was  successful  and  resulted  in  expected  changes  with 
significant  knowledge  and  skill  gain  hy  all  professors. 

After  the  semester  was  over,  the  professors  provided  the  following  responses  to  several  questions 
regarding  the  syllabus.  Generally,  they  felt  positive  about  the  syllabus  changes. 

See  their  responses  below  on  pages  5 and  6.  The  model  syllabus  is  presented  on  pages  7-13.  Finally,  the 
chart  on  pages  14-18  reveal  individual  component  evaluations  across  professors. 
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a.  Do  you  think  your  student  benefited  from  the  changes  in  the  syllabus? 

Yes 

I assume  so.  The  grading  scheme  was  much  clearer  and  well-defined.  Expectations  were  more  clear. 
Certainly,  yes 

Yes.  In  comparison  to  the  past  syllabus  for  this  course,  the  current  [new]  syllabus  is  much  more 
informative  in  terms  of  course  outcomes,  execution  schedule,  and  grade  distribution. 

Very  much  so. 

My  past  syllabus  for  this  course  didn’t  deviate  that  much  from  the  current  [new]  syllabus.  I would  say 
that  sections  of  learning  objectives,  schedule,  and  wording  in  general  were  rally  improved. 

Therefore,  I think  students  definitely  benefited  form  those  changes. 

Yes 

b.  Do  you  think  the  new  syllabus  communicated  your  expectations  to  them  more  clearly? 

Yes 

Yes,  definitely 

Yes.  There  were  immediately  aware  of  the  expectations,  including  performance  assessments. 

The  syllabus  provides  the  general  expectations  for  this  course.  While  the  performance  assessment 
rubrics  also  benefited  a lot. 

Yes,  definitely 

Yes.  I think  this  is  the  major  contribution. 

Yes,  especially  with  regards  to  objectives  (outcomes)  and  timeliness. 
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c.  Do  you  feel  that  the  guidelines  provided  by  the  syllabus  benefited  you  in  delivering  the  course? 
If  the  new  syllabus  benefited  you,  as  well  as  the  students,  please  note  how: 

Yes 

Yes.  More  rigorous  time  line  organized  the  course  better. 

Yes.  New  syllabus  is  much  more  detailed  and  organized.  I think  the  detailed  student  learning 
outcomes  were  most  beneficial. 

Yes,  this  provides  me  with  better  delivery,  planning,  and  getting  expected  response  from  students  (no 
more  surprises!) 

Not  really;  No  real  benefit  to  instructor. 

Yes,  I think  the  syllabus  is  good  navigation  map  for  the  semester. 

Yes,  it  helped  keep  everything  on  track. 

d.  What  were  the  students’,  if  any,  reactions  or  responses  to  the  syllabus? 

The  liked  the  detailed  information 
I got  no  reaction. 

I don’t  recall  any  direct  responses  or  reactions . . . .other  than  it  was  big. 

At  the  beginning  of  the  course,  I had  a brief  discussion  about  the  syllabus.  There  was  not  many  direct 
comments  about  the  syllabus.  However,  smooth  running  of  the  course  indicates  that  all  the  necessary 
information  were  available  within  the  syllabus. 

Positive,  from  students  that  used  it. 

I always  spend  the  first  hour  during  my  first  class  to  completely  review  the  syllabus  in  my  courses.  I 
have  received  comments  from  students  that  they  are  not  used  to  this  type  of  syllabus  and  their  sincere 
opinion  is  that  they  don’t  read  the  syllabus  that  much  (and  I know  this  is  the  case  most  of  the  time, 
reason  why  I state  in  my  syllabus  that  is  the  student’s  responsibility  to  keep  current  with  what  is 
scheduled  in  the  syllabus)  But  I think  this  is  the  change  of  culture  we  need  to  generate. 

At  first  they  were  scared  about  the  length.  After  going  through  the  syllabus  with  them,  they  realized 
all  of  the  information  contained  was  for  their  benefit  and  they  had  a positive  response. 
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B.lO.a:  Technology  496  - Industrial  Project  Management 

Prof:  Dr.  Scarborough  Grad.  Asst: Ph:  753-0210(Dr.  Scarborough)/1570(GA)  Off.Hrs: 

T12-3  Email: 

I.  Catalog  Course  Description:  Industrial  Project  Management  (3).  Basic  concepts,  principles,  and  skills 
of  project  management.  Designed  to  cover  a variety  of  types  of  project  management.  Emphasis  on  computer 
tools  and  project  management  techniques.  Analysis  of  case  studies.  Culminating  project  required. 

II.  Course  Purpose  & Objectives:  To  prepare  project  leaders  and  team  members  to  formally  initiate, 
execute  and  terminate  industrial  projects  effectively.  To  integrate  and  apply  knowledge,  skills,  and  abilities 
acquired  or  extended  during  students’  college  careers  (general  education  and  major)  and  work  experience  to 
research,  design,  build  and  finalize  a technical  project  within  a team  and  formal  project  environment. 

III.  Required  Text:  Project  Management.  Cleland  & Ireland,  2006  or  latest  edition.  Required:  Date 
book/Calendar  for  scheduling  and  notes;  Handout  packet. 

IV.  Pre-requisites:  Tech  265-Mfg.  Processes ;Tech  302-  Graphic  Pres.&  Comm.;Tech  395-Ind.  Data 
Processing;  Senior  Status 

Expected  Computer  Usage:  CAD,  MS  Office,  MS  Project,  CNC,  industrial  equipment,  or  other,  depending 
upon  semester/  team  project.  Required  Laboratory  Team  Project:  Changes  each  semester;  each  team  will 
engage  in  a complex  technical  project  with  specific  technical  standards  to  achieve,  e.g.  Go-kart,  3-car 
passenger  train,  hovercraft,  paddle  wheel  boat,  personal  transport  vehicle  etc.  Research,  design,  assembly  of 
electrical/mechanical  systems,  testing,  modifications/finalization  with  formal  documentation,  formal  team 
products  and  team  requirements.  See  requirements  section,  handouts,  and  rubrics. 


V.  Course  Requirements: 


Individual  Course  Requirements: 

Points: 

Team  Course  Requirements:  Points: 

Grading: 

Text  Project  (broken  into  sections/due  dates) 

7 

Team  Operations  Manual 

5 

Benchmark 

Project  Research 

5 

Community  Leadership  Service 

3 

A=98-100 

Project  Design 

5 

Project  & Articles 

Literature/Internet  Research  Tables  A & B 

7 

Team  Project  Plan 

5 

A=93-97 

(Proj  s/T  ms/Lead/Int  ’ 1/MCT  ms/MCLd) 

Team  Project  & Assessment 

7 

B=92-85 

Career  Project 

5 

(Final  Exam) 

C=  84-77 

Individual  Case  Study 

5 

*Peer  Assessment  Process 

5 

D=76-70 

Paper 

7 

& Team  Success  Assessment 

p=69-below 

Midterm:  Individual  Project  Plan 

7 

*Team  Member  Participation 

Software  Workshop/Test 

5 

(Ind.  Pts.  3/5) 

*Team  Participation  Awarded  by  Team 

3 

Team  Presentation  & Success 

5 

Project  Feedback  Logs 

1 

(Pinal  Exam) 

Individual  Portfolio  & Assessment  Process 

3 

Team  Project  Portfolio  & Website 

5 

(Final  Exam) 

Professor’s  Overall  Assessment 

Pinal  Exam  II:  TBD 

Ind.Presentation  within  Team  Presentation. 

5 

(if  needed  to  confirm  competencies) 

Total  Individual  Points  Possible 

65 

Total  Team  Points  Possible 

35 

Professor’s  Privilege 

See  Note  #2 
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VI.  Student  Learning  Outcomes 


Student  Learning  Outcomes 


lA/B.  Identify  and  describe  major 
problems,  issues,  concerns,  and  solutions 
(PICS)  that  relate  to  (a)  projects,  (b) 
project  management,  (c)  project  teams, 
and  (d)  project  leaders,  also  for  (e)  Int’l 
projects  and  (f)  multicultural  (MC)  teams. 

2.  Identify  and  describe  best  practices  for 
managing  projects  and  leading  teams; 
include  Int’l  teams  and  MC  teams. 


3a.  Demonstrate  effective  project 
management  of  a technical  project  using 
appropriate  PM  techniques,  tools,  and 
processes: 

a.  planning, 

b.  initiation, 

c.  execution, 

d.  termination 

e.  evaluation 

f.  problem  solving 

g.  leadership 

h.  financial  management 

i.  procurement  management 

j.  scheduling 

k.  MACE  process  and  procedures 

3b.  Design,  develop,  and  deliver: 

e.  executive  team  presentation 

f.  team  portfolio 

g.  team  website 

4.  To  integrate  mathematics,  the  sciences, 
English,  management,  technical, 
technological  systems  knowledge  and  skills 
to  accomplish  individual  and  team  project 
objectives: 

(a)  Design,  (b)  Build  a vehicle  to  technical 
specification  that  will  operate;  (c)  Solve 
technical  problems  encountered;  (d)  test 
and  evaluate  the  vehicle  for  meeting 
technical  specifications  and  standards 


5.  Develop  the  team  for  project  and  team 
work  by: 

a.  developing  a team  operations  manual 

b.  developing  peer  and  team  assessment 
system 

c.  creating  team  organization  & process 

d.  developing  team  project  plan 

6.  Demonstrate  effective  team 
performance  (hopefully  MC  team)  while: 

a.  engaged  in  a community  service  project; 
plan,  execute,  & report  relevance. 

b.  engaged  in  the  initiation,  planning, 
execution  and  termination  of  a technical 
project 

c.  engaged  in  course  Team  & Project 
activities 


Embedded  NIU  General  Ed 
Goals 

a.i.  communicate  clearly  in  English, 
demonstrating  ability  to  comprehend, 
analyze  and  interrogate  critically; 
ii.  communicate  in  a manner  that 
unites  theory,  criticism,  practice  in 
speaking  & listening; 
a.iv.  Aware  of  and  able  to  use 
various  resources,  including  modern 
technology 


a.  cultivate  habits  of  writing, 
speaking,  quantitative  reasoning  for 
continued  learning: 

a.i.  communicate  clearly  in  English, 
demonstrating  ability  to  comprehend, 
analyze  and  interrogate  critically; 
ii.  communicate  in  a manner  that 
unites  theory,  criticism,  practice  in 
speaking  & listening; 
a.iii. perform  basic  computations, 
display  facility  with  use  of 
quantitative  reasoning  in  forming 
concepts  for  analysis  and  in  problem 
solving,  and  interpret  mathematical 
models  and  statistical  info 

a. iv.  Aware  of  and  able  to  use 
various  resources,  including  modern 
technology 

b.  develop  an  ability  to  use  modes  of 
inquiry  across  a variety  of  disciplines 
in  the  physical  sciences, 
mathematics: 

b. iii.  demonstrate  ability  to  use 
scientific  methods,  theories  to 
science  phenomena; 

c.  develops  understanding  of 
discipline  interrelatedness,  applying 
that  knowledge  to  an  understanding 
of  important  problems  & issues. 


d.  develops  social  responsibility  & 
preparation  for  citizenship  through 
service  and  an  appreciation  of 
cultural  diversity. 


Embedded  NAIT/ABET 
Learning  Standards 

g. demonstrate  an  ability  to  communicate 
effectively  in  writing; 

h.  demonstrate  an  ability  to  communicate 
effectively  orally; 

m.  demonstrate  an  ability  to  . . . .utilize 
computer  applications  effectively; 
k.  demonstrates  a respect  for  diversity  and 
knowledge  of  contemporary  professional, 
societal  and  global  issues. 


a.  demonstrate  appropriate  mastery  of 
knowledge,  techniques,  skills,  and  modern 
tools  of  the  discipline; 

b.  demonstrate  ability  to  apply  current 
knowledge  and  adept  to  emerging 
applications  of  math,  science,  engineering 
and  technology; 

d.  demonstrate  ability  to  apply  creativity  in 
the  design  of  systems,  components  or 
processes  appropriate  to  program 
objectives; 

f.  demonstrate  ability  to  identify,  analyze, 
and  solve  technical  problems; 
g-h.  demonstrate  ability  to  communicate 
effectively  in  writing  and  orally; 

l.  demonstrate  commitment  to  quality, 
timeliness,  and  continuous  improvement; 

m.  demonstrate  ability  to  ....utilize 
computer  applications  effectively; 

o.  demonstrate  an  ability  to  manage 
projects,  industrial  systems,  lead  personnel 
effect. 

p.  demonstrate  an  ability  to  manage  and 
manipulate  industrial  systems; 

q.  demonstrate  knowledge,  strategies  and/or 
techniques  of  how  to  lead  personnel  and 
teams  effectively 


e.  demonstrate  ability  to  function 
effectively  on  teams; 

j.  demonstrate  ability  to  understand 
profess-ional,  ethical,  social  responsibilities; 

k.  demonstrate  respect  for  diversity, 
knowledge  of  contemporary  professional, 
societal  and  global  issues; 


Assessments/Rubrics 


Text  Project  or  Text  Test 
Research-  Literature/Intemet; 
Case  study; 

Group  analysis  process 
Formal  paper;  group  analysis 
1-5  minute  learning  papers; 
Individual  portfolios; 

Team  Project 
portfolio/website; 
Individual/team  presentations 
Team  participation  & Peer 
Assessment 

Team  Operating  Manual 
Individual  and  Team  Project 
Individual  & Team  project 
research 

Individual  & Team  project 
design 

Written  individual  & team 
plan(s); 

Technical  project  prototype 
product  produced  to  technical 
standards  and  specifications 
using  technical  processes 

Project  testing  & evaluation 
against  established  standards 
and  specifications  using 
formal  evaluation  tools  and 
procedures 

MS  Project  2003  test  and 
application  in  project 
planning,  execution, 
termination,  assessment  and 
evaluation 

MACE-Project  assessment 
(Plan  compliance  & 
adjustments) 

Individual  & Team  Logs 

Individual  and  Team 
Portfolio(s);  website(s); 
Individual  and  team 
presentations; 

Industrial  panel  evaluation 

Project  termination  with 
lessons  learned 
Project  evaluation  by 
industrial  panel 
Team  Operations  Manual; 
Team  Plan 

Team  presentation;  portfolios; 
website; 

Team  peer,,  team,  & conflict 
assessments/logs; 

Industrial  panel  evaluation; 
Formal  paper  ;5  minute 
learning  papers 
Team  success  rubric 
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VII.  Topics,  Class  Schedule  & Due  Dates 


Week/Date 

Topics 

Date 

Topics/Lab  Act. 

Assignment  Due  Dates 

1 

Course 

Intro 

9:30  Writing  Center  Requirements 
(Jacky) 

9:  45Career  Project  Intro  and  Requirements 
(Norwood) 

10:00  Team  Selection/Scheduling 
10:30  Course  Intro 
1 1 :30  Legacy  Group 

Teaming 

Team 

Assess. 

Team  Skills  Bank 
Finalize  Teams  & Schedules 

Plan  Team  Service  Project 

Project  Research  Review 

Due  1/20 

Writing  Center  Appointments 
Project  Research 
Bring  Planner 

Community  Service  Art.  & 
Plan  Duel/20  4:00pm 

Use  of  Planner  & The  Nature  of  Multitasking 
Project  Research  Assignment 

Schedule  Writing  Center 
NOW!!!! 

2 

Text  1-4, 19 

TEXT  Highlights 

Teaming 
*210  sched. 

Project  Design  Lab 

Writing  Center  Appts.  Due 
1/25 

Text  Proj.  1-4, 19,  20  due  1/25 
Industry  Case  ID  due  1/25 

Final  Project  Research  due 
1/27 

3 

Teaming 
Text  18  & HB 

Project  Teams:  hidden  agendas,  teamwork, 
effective  teams  & members,  member  roles  & 
responsibilities  - Peer  Assessment 

(Team  Packet  Required) 

Research 
Design 
*210  sched. 

Project  Design  Lab 

Text  Project  18,  21  due  2/1 
Project  Design  due  2/4  Friday 

4 

Teaming 
Text  20 

Project  Teams:  conflict  resolution,  decision- 
making, teams  in  trouble,  empowerment,  trust, 
recognition 

(Team  Packet  Required) 

Teaming 

Team  Manual  Lab 

Lit.  Research  Table  due  2/8 
Career  Project  due  2/11 

5 

Project 
Planning 
Text  11,6 

Project  Planning  - Section  I Rubric  & TEXT 
Vision,  Mission,  Intro,  Purpose,  Scope, 
Objectives,  Deliverables,  Charter,  Org.  Charts, 
Stakeholder  Analysis,  Com  Interface,  Project 
Review,  Change  Plan  [Paper  due] 

Teaming 

Research,  Case,  Paper,  Career  Validation 
Activity  - Group  Process 

Team  Manual  Lab 

Industry  Case  due  2/15 
Text  Proj.  6,  8, 11, 16  due  2/15 
Team  Manual  due  2/14 

6 

Project 
Planning 
Text  13, 

Project  Planning  - Section  II  Rubric  & TEXT 
Business  & Proj.  Success  Eactors,  SWOT 
Analysis,  Project  Constraints,  Risk  Analysis, 
Contingency  Plans  & Trade  Offs,  Statement  of 
Work,  Goals,  Work  Break-down  Structure 

Project 

Planning 

*210  sched. 

Project  Planning  - Section  II  Lab 

Community  Leadership 
Project  and  Articles  due  2/25 
Friday 

Paper  due  2/21 

7 

Project 

Planning 

Section  II  Rubric  & TEXT  (Continued) 

Life  Cycle,  Productivity  Plan, 

Quality  Standards  & Metrics, 

Project  Monitoring,  Assessment,  Control  and 
Evaluation,  Linear  Charts,  Resource 
Plan/Budget  MS  Project  - PM  Software 

Software 

Workshop 

*210  sched. 

MS  Project-PM  Software 

Text  Proj.  5,9, 12, 13, 14, 15 
due  3/1 

8 

Project 

Planning 

Section  III  Rubric  & TEXT 
Environmental/Safety  Plan,  Security  Plan, 
Documentation/Configuration  Mgmt.  Plan, 
Project  Divestment  & Termination  Plan 

Software 
Workshop 
*210  sched. 

MS  Project-PM  software 

Individual  Plans  dne  3/11 
Software  Test  due  by  3/9 

BREAK 

3/17 

BREAK 

9 

Project  Development  & Teamwork 

3/24 

Project  Development  & Teamwork  [Logs] 

Individual  Portfolios  due  3/22 
Team  Plans  dne  3/25 

10 

Project  Development  & Teamwork 

3/31 

Project  Development  & Teamwork  [Logs] 

Individual  Portfolios  due 

11 

Project  Development  & Teamwork 

4/7 

Project  Development  & Teamwork[Logs] 

12 

Project  Development  & Teamwork 

4/14 

Project  Development  & Teamwork[Logs] 

13 

Project  Development  & Teamwork 

4/21 

Project  Development  & Teamwork  [Logs] 

14 

Project  Development  & Teamwork 

4/28 

Project  Testing  and  Initial  Assessment 

Proj.  Test./Assess  due  4/28-29 

15 

[Final  Project  Assessment  & Grade] 

[Peer  Assessments  Executed  & Due] 
[Team  Member  Participation  Determined] 

5/5 

[Team  Presentations  8:30am-12:30] 
[Team  Portfolio/Website/Success  due] 

Team  Project  Assessment  due 
5/3 

Team  Pres./Port./Web.  Due  5/5 

16  Mav 

Finals 

Week 

5/12 

Final  Exam:  TBD 

If  needed  to  confirm  competencies 

VIII.  Course  Requirements  Explanation  --  Individual  Requirements 


Technical  Research  and  Design 

Project  Research:  Research  project  assigned.  More  information  about  this  research  will  be  provided  in  class.  However,  it  will  entail  an 
Internet/Literature  search,  possibly  interviewing  technical  experts,  local  or  suburban  vendors  or  manufacturers,  or  other  professors,  and/or  researching 
specific  technicalities.  It  will  also  include  research  of  all  properties  of  materials,  mathematics,  and  scientific  principles,  theories  involved  in  the 
technical  aspects  of  the  project.  Use  research  information  to  design  the  project.  See  Rubric.  Individual  and  Group  Process. 

Project  Design:  Students  will  design  and  prepare  visuals  and  working  drawings,  schematics,  etc.  for  the  project  using  prior  design  and  computer 
aided  drafting  or  mechanical  drawing  knowledge  and  skills.  See  Rubric.  Individual  and  group  process. 


Real  World  Validation  - Culminating  Paper 

Literature/Internet  Research  A:  Search  the  literature  (Internet)  on  project  management,  project  teams,  and  project  leadership;  identify  45  quality 
sources,  15  each  about  (a)industrial  projects, (h)  project  teams,  and  (c)project  leadership.  Develop  a literature/source  review  Table  summarizing 
what  the  literature/sources  revealed.  Topics  of  focus  should  be  the(l.)problems,  issues,  concerns,  (PICs)  difficulties  that  arise  on  projects  or  for  the 
teams  and  leaders  and  (2.)success  strategies  that  have  worked  for  projects,  project  teams  or  leaders  in  resolving  the  problems/issues.  There  must  be 
45  sources;  these  must  be  from  major  recognized  journals  or  books  on  the  topics.  You  may,  however,  include  up  to  five  non-traditional  sources,  e.g. 
Internet  sources  from  industrial  groups,  project  teams,  etc.  Sources  must  show  depth  in  content;  short  “briefs”  are  not  acceptable.  Copy  all  sources  if 
not  books  on  diskette  or  CD  rather  than  hardcopies.  See  Table  Format  and  Rubric.  Group  Process-Be  prepared  to  discuss;  thus,  if  no 
hardcopies  available  for  reference,  you  need  to  know  the  material  well.  **Create  tables  that  are  categorized,  numbered  and  reveal  in-depth 
information  with  solutions.  May  use  for  your  paper. 

Literature  A + B = Total  Table  (See  Rubric) 

Literature/Internet  Research  B:  Also,  research  (a)international  projects,  (b)multicultural  teams,  and  (c)international  project  leadership  with  a 
multicultural  team;  identify  15  (5  for  each  topic)  Internet  and/or  literature  sources  that  discuss  (l.)problems,  issues  and  (2.)  best  practices,  benefits, 
successes  of  multicultural/international  projects,  teams,  and  project  or  team  leadership.  Summarize  the  information  learned  by  organizing  it  into  a 
Table  identifying  the  source  author,  title,  main  points  on  problems,  issues,  and  benefits  and  your  comments.  See  Rubric.  Individual/Group  Process- 
Be  prepared  to  discuss;  thus,  your  if  no  hardcopies  available  for  your  reference,  you  need  to  know  the  material  well.  **Create  tables  that  are 
categorized,  numbered  and  reveal  in-depth  information  with  solutions.  May  use  for  paper. 

Industrial  Case  Study:  Identify  a company  that  will  allow  you  to  visit  and  interview  an  industrial  project  team.  Interview  a project  leader  or 
manager  and  at  least  three  project  team  members  or  2 project  leaders  and  2 project  team  members.  (l.)Ask  them  to  identify  all  problems,  issues, 
concerns,  (PICs)  or  difficulties  encountered  on  the  project,  about  the  project,  team.,  and  project  leadership.  Have  them  explain  in  detail;  (2.)then,  also 
ask  them  what  strategies  are  successful  for  projects,  teams,  and  project  leaders.  Create  a table  of  questions  and  responses  and  present  what  was 
learned  as  “real-time”  research.  See  Rubric  and  Format.  Individual  and  Group  Process.  Incorporate  the  results  into  your  paper. 

Formal  Paper:  Meet  with  WC  tutor  to  orgauize  paper.  Develop  a paper  about  projects,  teams  and  project  leadership;  develop  the  issues 
and  solutions  in  greater  depth;  draw  conclusions  and  describe  effective  project  management,  effective  project  teams,  and  effective  project  leadership. 
What  strategies,  techniques,  processes  should  be  used  to  have  a more  successful  project,  team,  or  leader/leadership  process?  End  with  very  specific 
recommendations  to  guide  your  project  team  on  each  of  the  3 primary  topics.  Then  include  a section  on  how  international  projects  and  multi-cultural 
teams  differ,  what  additional  concerns,  problems,  and  issues  occur  when  operating  internationally  with  diverse  cultures.  Make  recommendations  for 
successful  international  projects  and  on  how  to  be  a more  effective  leader  of  multicultural  teams.  Sixty  (60)  sources  required  (45  -l-  15).  These  60 
sources  may  or  may  not  be  the  same  ones  that  you  identified  for  the  literature  review  table.  **Incorporate  the  results  of  industrial  case 
study  into  your  paper  as  well.  Use  the  APA  writing  style  manual.  Identify  all  sources  in  the  paper’s  text  and  in  References  Cited  using  the  APA 
style  format.  Writing  skills  are  seriously  graded  on  this  product.  See  Writing  Rubric,  Paper  Outline  & Rubric.  Individual  and  Group  Process. 


Project  Planning  - Midterm  Exam 

MS  Project  Software  Workshops/Test:  Participate  in  the  software  workshop(s).  Complete  Test.  MS  Project  documents  required  in  PLANs. 
Midterm  - Individual  Project  Plan:  Use  the  outline  & rubric  provided  as  a guide,  develop  a detailed  project  plan.  The  plan  will  not  be  accepted 
unless  every  category  is  complete.  Reference  the  text,  other  sources  in  the  library  or  through  the  Internet,  or  sources  listed  on  the  course  syllabus.  All 
members  of  a team  must  have  their  plan  in  and  graded  before  they  will  be  approved  to  work  on  the  “team”  plan.  This  is  another  product  where 
writing  will  be  graded  seriously.  This  is  technical  writing  which  is  different  than  the  narrative  or  prose  approach  used  in  the  above  assignments.  See 
Plan  Outline/Rubric.I/GP 
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Employment 

Career  Project:  a)  Interview  Mr.  Norwood,  the  CEET  Career  Planning  & Placement  specialist,  on  the  assigned  topic;  engage  in  group  process. 
Document  findings  as  assigned  (TBD);  b)Research  jobs/positions/career  in  project  management;  Bring  in  copies  of  10  position 
announcements  which  review  expectations,  required  knowledge,  skills,  background  for  those  seeking  to  become  project  managers,  team 
leaders,  or  project  team  members;  c)  Design  and  develop  a resume  to  use  to  seek  such  a position,  but  also  make  it  applicable  for  other  industrial 
technology,  management,  engineering,  etc.  positions.  Have  it  reviewed  and  approved  by  Mr.  Norwood  for  inclusion  into  personal  496  portfolio.  Mr. 
Norwood  will  grade  this  project. 

Individual  Portfolio;  This  portfolio  has  a somewhat  different  focus.  Although  it  may  contain  everything  in  the  team  portfolio  for  job-seeking 
purposes,  it  must  also  include  all  individual  work,  including  Writing  Center  Reviews  and  multiple  iterations  of  particular  products.  Use  Course 
Requirements  list  on  Course  Syllabus  (above)  and  Team  Portfolio  Rubric  to  determine  what  is  to  be  included.  You  will  participate  in  assessment 
activities  throughout  the  semester,  including  analysis  and  reflections  about  what  your  strengths  and  weaknesses  are  and  what  you  can  do  to  improve 
or  continue  well.  The  portfolio  must  be  professionally  presented,  e.g.  typed  tabs,  etc.  Final  Reflections  at  end  of  semester/questions  to 

nncwpr.. 


Team  Requirements; 

Community  Service  Project:  Each  team  has  to  research,  determine,  plan  and  execute  an  8 hour  service  project.  Research  one  article 
per  team  member  on  the  benefits  of  community  service  and  leadership  by  local  industrial  personnel.  Generate  a brief  team  plan  of 
what,  who,  when  and  where.  It  should  include  a goal,  operational  objectives,  expected  outcomes  and  benefit  to  group  served.  Prepare 
an  informal  presentation  about  what  you  learned,  how  you  felt  and  your  potential  future  in  community  service.  See  Rubric. 
Individual/  Group  Process 

Team  Process 

Team  Manual:  The  team  manual  includes  all  team  operational  policies  and  procedures,  the  team  problem-solving  process,  communication  strategy 
and  procedures,  decision-making  process,  authority  linear  charts,  team  roles  and  responsibilities,  etc.  The  team  is  to  provide  evidence  that  it  operated 
using  the  team  manual  as  its  stmcture,  process  and  guiding  document.  See  Outline/Rubric.  Group  Process. 

Included  in  the  Team  Manual  are  the  following  critical  components,  plus  others:  See  Team  Manual  Outline/Rubric.  Group  Process. 

Team  Skills  Bank:  Each  team  will  prepare  a team  skills  bank  that  identifies  all  individual  talent,  skills,  knowledge  that  each  team  member 
brings  to  the  project.  This  bank  will  be  used  to  organize  the  team,  project,  work  packages  and  deliverables.  Group  Process. 

Team/Project  Charter,  Logo,  Company  & Project  Organizational  Chart:  Each  team  will  create  an  official  charter,  identify  a team  logo, 
and  design  an  organizational  chart  for  their  company  and  their  team/project.  Group  Process. 

Team  Assessment  Inventory(ies):  Each  team  will  design  and  develop  a peer  and  team  status  inventory  to  use  to  monitor  team  process;  they 
will  also  adapt  a conflict  management  inventory  to  use  to  monitor  the  team  conflict  resolution  process.  The  information  gained  from  using 
these  inventories  will  be  used  to  build  and  strengthen  the  team  and  to  identify  and  solve  team  issues  or  problems.  Growth  and  development 
should  be  an  outcome  of  using  these  instruments.  Each  team  must  produce  a report  of  results  from  using  these  instruments  and  assessment 
process  twice  during  the  project  period. 

Team  Project  Plan:  Each  team  must  write/develop  a team  plan;  however,  the  team  may  not  begin  on  the  team  plan  until  all  individual 
plans  are  graded  and  returned.  The  plan  outline  is  the  same  as  the  individual  plan.  Each  team  must  produce  a plan  for  the  technical 
project  assigned  and  use  the  plan  as  a compliance  document  to  monitor,  assess,  control  and  evaluate  the  project.  See  Outline/Rubric. 
Group  Proc. 

Logs:  Periodically  you  will  be  asked  to  complete  a log  about  how  you  feel  the  team  and  project  are  progressing.  Completed  and  turn  in. 

Individual  Component  of  Team  Presentation;  Speaking,  non-verbal  communication,  presentation  skills,  content,  grammar/wording  visuals, 
style,  organization,  use  of  technology,  humor,  etc.  graded  individually  during  team  presentation.  Remember  that  each  team  member  must  demonstrate 
speaking  and  presentation  skills.  Teams  could  acquire  the  full  point  value,  but  individuals  will  be  assessed  on  their  individual  performance  as  well. 
Professional  dress  required.  See  Presentation  Outline/Rubric. 

Team  Participation  Points  Awarded  by  Team:  Each  team  member  will  be  allocated  points  for  team  participation.  Teams  will  award  points  to 
team  members  for  quality  of  work  and  participation.  Points  will  serve  to  “grade”  participation.  Dr.  Scarborough  validates  that  the  distribution  is 
appropriate  for  participation  observed.  Full  participation  is  expected  of  each  team  member.  Tardiness  or  absences  from  team  meetings,  class,  labs  are 
not  acceptable  behaviors.  You  will  be  asked  to  explain  to  the  class  openly  why  you  are  late  or  absent  and  points  will  be  deducted.  See  Rubric. 
Individual/  Group  Process 
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Final  Exam 

Team  Project:  Each  team  will  be  responsible  for  designing  and  developing  a technical  project.  You  will  generate  technical  standards  to 
achieve  and  the  metrics  to  use  to  measure  the  standards  achievement  level.  The  project  must  “function”  or  “work”  to  be  accepted  for  a grade.  It 
must  meet  the  standards  at  the  level  described  in  the  team  plan  using  the  metrics  predetermined.  Every  team  member  must  have  major  project 
role  and  responsibilities.  The  team  must  complete  the  project  by  the  deadline  on  the  syllabus.  The  project  is  the  “vehicle”  providing  evidence  of 
high  performance  teaming  and  project  management  as  well  as  the  knowledge,  skills,  and  abilities  from  academic  career  and  work  experience. 
Team  derived/Professor  approved-  predetermined  -Standards/metrics  = grading  Rubric.  Group  Process. 

Team  Portfolio:  The  portfolio  is  the  culminating  documentation  of  all  project  and  team  work.  It  must  include  information  on  every  topic  listed 
in  the  outline/rubric.  It  should  include  pictures,  mechanical  drawings,  etc.  and  be  professionally  produced  in  hard-copy  form.  An  operator’s  and 
maintenance  manual  must  be  developed  and  included  for  the  technical  project  (product).  See  Rubric.  Group  Process. 

Team  Website:  Each  team  is  to  design  and  produce  a team  web-site  which  will  serve  as  an  electronic  portfolio.  This  website/  portfolio  must  be 
presented  during  the  team  presentation.  The  outline  is  the  same  as  the  hard-copy  portfolio.  A CD  must  be  included  in  the  hard  copy  of  the  team 
portfolio.  See  Rubric.  Group  Process. 

Final  Team  Presentation:  Each  team  is  to  professionally  present  their  project,  portfolio/website  and  information  for  each 
category  on  the  presentation  outline.  This  is  a formal  presentation  where  communication  skills,  presentation  skills,  etc.  will  be 
graded.  Professional  dress  required.  An  industrial  panel  will  observe  the  presentations.  Presentation  CD  must  be  in  Portfolio. 

See  Outline/Rubric. 


VIII.  Cheating: 

Cheating  is  unacceptable;  refer  to  the  NIU  Judicial  Code;  any  students  cheating  will  be  dismissed  from  the  course  immediately. 

IX.  Academic  Misconduct:  Refer  to  the  NIU  Judicial  Code;  Immediate  and  appropriate  actions  will  occur  for  any  students  behaving 
inappropriately,  e.g.  cheating,  will  be  dismissed  from  the  course  immediately. 

X.  Professor’s  Role:  This  course  involves  the  professor  and  graduate  assistant  in  a variety  of  roles;  the  professor  will  provide  a scenario, 
objectives,  and  standards  and  then  guide,  coach,  and  direct  most  of  the  time,  however,  there  will  be  some  lectures.  This  course  is  performance 
based,  thus,  there  are  usually  no  traditional  objective  tests.  There  are  subjective  tests  in  the  form  of  the  5-10  minute  learning  papers,  essays  and 
the  text  project  to  determine  concept  attainment.  Students  will  construct  knowledge/skills  while  engaged  in  learning  & performances. 

Assessment  will  occur  as  learning  occurs. 

XI.  Professor’s  Notes: 

1.  Unexcused  absences  could  result  in  one  letter  grade  reduction  each  (7pts).  Class/lab/  team  meetings/work  sessions  attendance  mandatory. 
Tardiness  unacceptable.  Door  may  close  when  class  begins;  late  admittance  may  not  be  permitted  according  to  prof.’s  prerogative. 
Unexcused  class/team  tardies,  I point  per  30  minutes  IF  you  are  allowed  in  and  door  is  open;  don’t  count  on  door  being  open. 

2.  The  professor  reserves  the  right  to  determine  the  final  grade  in  the  case  of  a student  who  does  not  perform  on  the  team. 

3.  Unexcused  late  projects/assignments  will  result  in  point  reduction,  2 points  per  day  late. 

4.  Dress  code:  no  hats  in  lab  ever!  Professional  dress  required  for  final  presentation. 

5.  Monitor  language  in  class/lab  at  all  times;  good  grammar  and  communication  skills  expected  at  all  times;  professional  language  expected. 

6.  Students  are  required  to  see  the  Writing  Center  tutor  for  all  written  assignments  until  approved  otherwise,  at  least  2 visits  per  assignment;  3 
visits  required  for  paper.  (I)  Meet  once  to  design  paper,  then  meet  with  draft  in  hand  (2-3)  twice  and  rewrite.  An  appointment  to  plan  the 
written  assignment  with  no  draft  for  review  would  still  require  2 other  visits  for  all  other  assignments. 

7.  Unannounced  individual  portfolio  checks  throughout  course;  5 point  penalties  for  portfolios  not  up  to  date  each  time. 

8.  No  cell  phone  ringers  in  the  class  or  lab  at  any  time;  5 points  deducted  for  in-class  interruptions.  See  professor  exception  approval. 
9.Students  can  not  pass  class  without  AUU  assignments  turned  in.  Student  will  receive  an  I (incomplete)  until  all  assignments  are  turned 
in.  Penalties  may  occur  for  grades  of  Incomplete. 

XII.  Support  Services  Available  for  Students:  The  NIU  writing  center  provides  tutoring  for  writing.  Students  in  this  class  are  required  to  use 

that  service  for  all  written  assignments;  each  writing  assignment  requires  two  visits/critiques  and  rewrites  before  assignment  can  be  handed  in  to 
professor.  Tutor  signatures  and  forms  are  required  to  be  turned  in  with  written  products.  Math  and  science  tutors  available  in  College.  NIU 
accommodations  for  any  student  with  special  needs.  See  professor  individually. 

XIII.  References  on  reference  in  Founders  Uibrary  on  NIU  main  campus:  Kerzner.  Smith.  Project  Management  & Teamwork. 
McGrawHill.  Angus,  Gundersen,  Cultinane.  Planning,  Performing  and  Control-  ling  Projects.  Prentice  Hall.  2000;  Dinsmore.  Human 
Factors  in  Project  Management.  Dinsmore.  Project  Management.  Thomsett.  The  Uittle  Black  Book  of  Project  Management. 
AMACOM;  Kerzner,  Thamhain.  Project  Management  Operating  Guidelines.  VNR.;  Rosenau.  Successful  Project  Management.  VNR.; 
Weiss,  Wysocki.  5-Phase  Project  Management.  Addison  Wesley;  Cleland,  Gareis.  Global  Project  Management  Handbook. 
McGrawHill.;  Miller.  Visual  Project  Planning  & Scheduling;  Barkley,  Saylor.  Customer  Driven  Project  Management.  McGrawHill; 
Uewis.  Mastering  Project  Management.  McGrawHill;  Forseberg,  Mooz,  Goterman.  Visualizing  Project  Management.  Wiley;  Dinsmore. 
Winning  in  Business  With  Enterprise  Project  Management.  AMACOM;  Graham,  Englund.  Creating  an  Environment  for  Successful 
Projects.  JoseyBass;Gray,  Uarson.  Project  Management.  McGraw  Hill.  2000.;  Cleland.  Project  Management.  McGrawHill;  Meredith, 
Mantel.  Project  Management.  Wiley.  2000;  Uewis.  Team-based  Project  Management.  AMACOM;  Kliem,  Eudin.  Project 
Management  Practitioner’s  Handbook.  AMACOM;  Kerzner.  In  Search  of  Excellence  in  Project  Management.  ITP  VNR;  Ruskin, 
Estes.  What  Every  Engineer  Should  Know  About  Project  Management.  Dekker;  Buttrick.  The  Project  Workout.  FT  Pitman; 
Rowenau.  Project  Management  for  Engineers.  Ran  Nostrand  Reinhold;  Cleland.  Field  Guide  to  Project  Management.  ITP  VNR. 
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XIV.  Course  Requirements  Check  Off 

Individual  Contributions:  Benchmark=98-I00  (This  means  that  you  set  the  standard  for  others.) 

(7)  Text  Project  A=93-100  points 

B=92.9-85  points 

(5)  Project  Research  C=84.9-77  points 

D=76.9-70  points 

(5)  Project  Design  F=Below  70  points 

(7)  Literature/Internet  Research  Tables  A & B 

(5)  Career  Project 

(5)  Industrial  Case  Study 

(7)  Paper 

(7)  Midterm:  Individual  Project  Plan 

A). 

(5)  Software  Workshop/Test 

(1)  Project  Feedback  Logs 

(5)  Professor’s  Overall  Assessment 

(65)  Total  Individual  Points  Possible 

Team  Contributions: 

(5)  Team  Manual 

(3)  Community/Leadership  Service  Project/ Articles 

(5)  Team  Project  Plan 

(7)  Team  Project  & Assessment  (Final  Exam) 

(5)  *Peer  Assessment  Process/Team  Success 

(5)  *Team  Member  Participation 

(5)  Team  Presentation  & Success  (Final  Exam) 

*Individual  Presentation  (in  Team  Pinal  Presentation) 

(5)  Team  Project  Portfolio  & Website 

(35)  Total  Team  Points  Possible 


IP  a teams  goal  is  to  be  a Benchmark 
Team,  where  the  team  best  exemplifies 
the  course's  highest  standards  for 
teams . . . where  the  team  sets  the  standard 
for  other  teams,  then  every  team 
Member  in  that  team  must  maintain 
98%  or  ultimately  98  points  each  for  the 
course. 


Note:  To  keep  track  of  your  progress, 
add  the  possible  points  of  work  to  date; 
then  figure  the  percentage,  e.g. 

Text  (7)+P.Research(5)+P.Design(5)=17 
.93  X 17  = 15.81  = lowest  possible  score 
or  point  value  to  maintain  an  A(lowest 

IP  your  goal  is  to  be  a Benchmark 
Student,  where  your  work  best 
exemplifies  the(confirmed  by  Professor) 
course's  highest  standards  ...where  you 
set  the  standard,  then  you  must  maintain 
no  lower  than  98%  or  ultimately  98 
points  for  the  course. 
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Table  B.10.1:  Syllabi  - Fall  2005  Fall  2006  (Syllabus  Rubric,  Scarborough,  2006) 


Syllabus 

Components 

RM 

RR 

AA 

BT 

IM 

BC 

AG 

05 

06 

05 

06 

05 

06 

05 

06 

05 

06 

05 

06 

05 

06 

Faculty/GA 
Contact  Info 
Crse.-Off.Hrs. 

J 

+ 

J 

+ 

J 

+ 

J 

+ 

J 

+ 

J 

+ 

J 

+ 

I. 

a.  Course 
Description 

J 

+ 

J 

+ 

J 

+ 

J 

+ 

J 

+ 

J 

+ 

J 

+ 

b.  notes  lab 

D 

? 

J 

? 

J 

? 

0 

7 

D 

7 

0 

? 

D 

7 

II. 

Course 

Purpose 

D 

+ 

D 

+ 

D 

+ 

0 

+ 

D 

0 

0 

D 

D 

D 

III. 

Requirements: 
Text,  Plan 
book,  handout 
put,  other,  etc. 

J 

+ 

J 

+ 

J 

+ 

J 

+ 

+ 

J 

+ 

J 

+ 

IV. 

a.  Pre- 
requisites 

J 

+ 

J 

+ 

J 

+ 

J 

+ 

J 

By 

topic./ 

+ 

By  topic 

J 

D 

J 

+ 

b.  Computer 

Technology 

Use 

J 

+ 

D 

0 

J 

+ 

0 

+ 

D 

D 

J 

+ 

D 

D 

Some- 

what 

Inherent 
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Syllabus 

Components 

RM 

RR 

AA 

BT 

IM 

BC 

AG 

V. 

a.  St.  Learning 

Outcomes- 

Objectives 

*Objs.  05 

Most  called 
Outcomes,  Objs. 

y 

ABET 

Labeled 

CDurse- 

Pregpa 

m 

Dutcom 

es 

+ ReI. 
Rating 

+ 

ABET+R 
el.  Rat. 

Dept. 

Ed. 

Objs. 

y 

ABET 

Labeled 

CDurse- 

Pragra 

m 

nutcam 

es 

+ Rel. 
Ratin 
g 

+ 

ABET 

y 

ABET/NAIT 

LB 

+ 

ABET/ 

NAIT 

y 

ABET/NAIT 

LD 

St.  LO 
M.Verbs? 

+ 

ABET 

/NAIT 

y 

ABET 

Mixed 

style 

+ 

ABET 

y 

ABET 

+ 

ABET 

y 

style 

confus 

ing 

+ 

ABET 

b.  Course 
Objectives 
(NR-Not 
requested) 

y 

Ex. 

Verbs 

NA 

y 

M.Verbs 

7 

+ 

In  Purpose 

y 

M.Verbs? 

NA 

NR 

NA 

P 

NA 

y 

M. 

Verbs 

9 

NA 

y 

M. 

Verbs 

9 

NA 

b.  Embedded 
Gen  Ed  Goals 

0 

D 

D 

+ 

D 

+ 

D 

+ 

D 

+ 

ABET 

D 

D 

D 

+ 

c.  Embedded 
ABET/NAIT 
Standards 

y 

+ 

y 

+ 

y 

+ 

y 

+ 

y 

+ 

ABET 

y 

+ 

y 

+ 

d.  Assessment 
Connections 

0 

D 

D 

+ 

D 

P should 
be 

itemized 

D 

+ 

p 

+few 

+ 

D 

D 

D 

+ 

VI. 

Course  Outline 
Schedule 
Assignments 
Due  Dates 

y 

P 

No 

Assign. 
No  Due 
Dates 

y 

No 

Schedule 

+ 

y 

No  schedule 

+ 

p 

Readings 

+ 

P 

Topic 

s 

No 

sched 

ule 

+ 

p 

Topic 

s 

+ 

p 

+ 
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Syllabus 

Components 

RM 

RR 

AA 

BT 

IM 

BC 

AG 

VII. 

a.  Course 
Requirements- 
assessments/ 
score  values 

b.  Individual 
vs.  Team  Req. 

D 

p 

Read- 

ings; 

soft- 

ware 

P 

+ 

D 

D 

D 

+ 

0 

D 

D 

+ 

D 

D 

p 

Projects, 

Homework 

0 

+ 

D 

D 

+ 

0 

D 

+ 

+ 

D 

D 

D 

D 

VIII 

Policies  (e.g., 
cheating,  lap- 
tops, conduct, 
book  bags, 
calculators, 
due  dates,  etc.) 

J 

+ 

D 

+ 

J 

Attendance 

Homework 

Misconduct 

+ 

D 

D 

+ 

J 

+ 

D 

+ 

IX. 

Professor’s 

Role 

(Responsibiliti 

es) 

D 

D 

D 

+ 

D 

+ 

0 

D 

D 

0 

0 

+ 

+Student’ 
s Role 

D 

D 

X. 

Professor’s 

Notes 

J 

D 

D 

+ 

D 

+ 

0 

D 

D 

0 

D 

+ 

D 

+ 
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Syllabus 

Components 

RM 

RR 

AA 

BT 

IM 

BC 

AG 

XL 

Support 

Services 

0 

D 

D 

D 

P 

Special 

assistance 

D 

0 

D 

D 

0 

D 

+ 

D 

D 

XII. 

Course 

References 

Resources 

J 

Libra 

ry 

+ 

D 

0 

J 

D 

D 

D 

J 

+ 

J 

+ 

D 

D 

XIII 

Course 

Requirements 

Explanation 

D 

Lists 

only 

D 

Lists 

only 

D 

Lists 

only 

0 

Lists- 

LuU 

PAs 

D 

Lists 

only 

D 

Lists 

Brief 

Descrip- 

tions 

D 

Lists 

only 

XIV. 

a.  Course 
Requirements 
Check  Off/ 
Points  for  Self 
Tracking 

D 

+ 

% 

Categor 

ies 

D 

+ 

D 

+ 

0 

+ 

D 

0 

D 

+ 

D 

+ 

b.  Grading 

Values 

Explanation 

J 

% 

Pt. 

vaiues 

+ 

% 

Points 

J 

% 

+ 

% 

Benchmarks 

J 

% 

+ 

Points 

J 

% 

+ 

Points 

D 

+ 

% 

J 

% 

+ 

% 

Points 

J 

% 

+ 

Points 

O 

T 

H 

E 

R 

Fin. 

Rpt. 

Far. 

Research 

Explana- 

tion 

El. 

Expecta- 

tions 

Parti- 

cipa- 

tion 

Q 

Level 

N 

G-N 

N 

E-S 

N 

G-N 

N 

G-N 

N 

G-N 

N 

G-N 

N 

G-N 

iBgBnd:  J = Dobb  rathBr  wall 

Buality  LbvbI:  S =SupBrSyllabus 


’ = Paptia 

E = Excallant  according  to  Benchmark  B Literature 


NA-Nat  applicable 

G = goad/ adeguate  N = needs  important  improyements 
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MODELS  AND  STYLES  OF  TEACHING  SUMMARY 

(See  Tables  in  Portfolio  Section  B.5.b;  also,  B.ll.a,  b,  c,  and  Section  A. 5) 

Jule  Dee  Scarborough,  Ph.D. 


Professors  engaged  in  the  consideration  of  teaching  models  in  several  program 
components  throughout  the  program.  Initially,  in  analyzing  the  2005  courses,  they 
considered  what  teaching  models  were  being  used  during  those  courses.  Later,  they 
studied  them  more  deeply  to  determine  or  select  a few  to  use  in  their  redeveloped  courses 
for  the  2006  experimental  research  semester,  known  as  the  2006  courses.  The  teaching 
model  goal  for  each  professor  was  to  broaden  his/her  repertoire  of  teaching  models  and 
select  some,  other  than  lecture,  to  use  in  newly  developed  2006  courses.  A common 
model  chosen  by  all  the  professors  was  cooperative  learning;  others  were  selected  as  well 
and  differed  across  professors. 

Course  Analysis 

The  initial  consideration  of  teaching  models  was  during  the  Course  Analysis  program 
component.  Professors  analyzed  their  2005  courses  to  identify  their  current  use  of  the  24 
teaching  models  presented  by  Joyce,  Weil,  and  Calhoun  (2004).  (See  the  GAPS  Analysis 
results.  Portfolio  Section  B.5.3)  Generally,  they  all  realized  that  their  reliance  on  lectures, 
where  students  are  passive  (Dale’s  Cone,  1964)  and  the  professors  “impart”  course 
content,  was  far  too  frequent  and  was  their  primary  teaching  model.  Once  they  realized 
the  number  of  models  available  from  which  to  choose,  they  were  more  than  willing  to 
expand  their  repertoire  of  teaching  models.  The  GAPS  Analysis  Summary  reveals  that 
the  professors  are  willing  to  consider  using  many  of  the  other  models  presented  by  Joyce 
et  al.  (2004).  Once  they  realized  their  model  options,  the  professors  were  positive  about 
choosing  some  new  ones  to  try  during  the  experimental  research  semester  in  2006. 

Consideration  of  teaching  models  was  triangulated  with  teaching  styles  and  student 
learning  styles.  During  this  analysis,  professors  also  studied  their  2005  courses  to 
determine  which  teaching  styles  they  used  in  the  courses.  During  this  initial  exposure, 
they  were  introduced  to  Mosston  and  Ashworth’s  (1990)  styles  and  found  that  the 
primary  style  used  was  abstract  conceptualization.  They  realized,  once  again,  that  there 
was  an  opportunity  to  increase  their  repertoire  of  teaching  styles  and  were  willing  to 
make  some  new  choices  to  implement  in  the  2006  courses.  Teaching  models  and  styles 
go  in  tandem  with  each  other,  so  their  willingness  to  expand  their  use  of  models  for  both 
was  reinforcing. 

Finally,  the  third  aspect  that  completes  the  triangle,  related  to  the  relationship  between 
teaching  and  learning,  was  the  learning  styles  they  made  possible  for  students  to 
experience  as  a result  of  the  teaching  models  and  styles  used.  To  learn  first  hand  about 
learning  styles,  they  analyzed  their  own  learning  style  using  the  Kolb  Inventory.  We  also 
discussed  how  the  faculty  development  program  had  presented  opportunities  for  them  to 
observe  or  experience  a wide  range  of  teaching  models  and  styles  while  learning,  as  we 
modeled  what  we  were  asking  them  to  use  in  their  own  classrooms.  The  modeling  also 
provided  different  learning  style  opportunities  for  them  as  they  progressed  through  the 
faculty  development  program.  After  they  studied  their  own  individual  Kolb  (1984) 
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learning  style  profile  and  engaged  in  a round  table  of  discussion,  they  better  understood 
how  to  broaden  the  learning  style  experiences  for  their  students.  Three  professors  decided 
to  formally  use  learning  style  inventories  with  their  students.  The  results  are  presented  in 
the  GAPS  Analysis  Summary  in  Portfolio  Section  B.5.3,  and  it  is  copied  into  this  section 
for  reader  convenience.  The  professors  realized  that  the  primary  learning  style  possible  as 
a result  of  the  teaching  models  and  styles  used  in  their  2005  course  was  abstract 
conceptualization  (Kolb,  1984).  Thus,  the  realized  the  need  to  broaden  student  learning 
style  opportunities  in  the  2006  courses;  They  were  very  willing  to  structure  the  course 
and  teaching  models  and  styles  to  increase  learning  style  opportunities  in  the  2006 
course. 

2006  Course  Development 

Once  they  completed  the  new  student  learning  outcomes,  the  new  tests  and  performance 
assessments,  and  the  new  syllabus  draft,  each  professor  then  engaged  in  a deeper  study  of 
the  teaching  models  by  using  the  following  worksheet  with  the  Joyce  et  al.  (2004)  book 
to  more  closely  examine  each  model.  They  were  also  provided  with  more  perspectives  on 
styles,  Grasha  (1996),  further  realizing  that  they  relied  on  the  “expert”  style  as  presented 
by  Grasha.  They  were  also  introduced  to  another  perspective  on  learning  styles  by  Felder 
(1988).  Two  professors  chose  to  use  the  Kolb  Inventory,  and  one  chose  the  Felder 
Inventory.  This  study,  or  deeper  examination,  of  the  teaching  models  was  followed  by 
questions  and  discussion.  Once  they  felt  comfortable,  they  individually  made 
instructional  decisions  about  which  teaching  models  and  styles  they  wanted  to  use  in  the 
2006  course  that  would  result  in  students  experiencing  the  whole  range  of  learning  styles. 
The  professors  hoped  to  stimulate  opportunities  for  students  to  engage  in  a broader  range 
of  learning  styles,  so  each  student  would  have  the  opportunity  to  learn  in  his/her  learning 
style  “comfort  zone,”  while  also  broadening  his/her  capabilities  across  learning  styles. 

Data  is  presented  in  several  ways.  The  Gaps  Analysis  data  presents  both  the  2005 
responses  resulting  from  the  course  analyses  as  well  as  the  data  on  their  responses  about 
what  models,  styles,  and  learning  styles  actually  occurred  throughout  the  2006 
experimental  course. 

The  process  was  a simple  one.  They  reviewed  the  worksheets  from  the  2005  course 
analysis  indicating  which  models,  styles,  and  learning  styles  they  felt  occurred.  In 
addition  to  the  reconsiderations  of  the  2005  worksheets,  they  also  responded  to  a list  of 
models,  styles,  and  learning  styles  with  brief  descriptions  and  indicated  which  ones  they 
felt  were  used  or  experienced  in  the  2006  course.  And,  finally,  they  reviewed  the  study 
forms  they  completed  when  more  deeply  studying  the  models  and  noted  which  ones  they 
felt  were  used.  Therefore,  after  teaching  the  2006  courses,  they  came  back  to  informally 
analyze  what  they  felt  occurred  using  a variety  of  formats.  The  picture  became  clear  that 
they  did  try  many  new  models  and  styles  that  probably  stimulated  students  to  engage 
across  the  learning  styles.  In  one  semester,  we  felt  they  accomplished  a great  deal.  Are 
there  many  more  models  and  other  styles  they  can  introduce  to  broaden  student  learning 
style  experiences?  Yes,  definitely,  and  they  will  also  become  more  expert  in  using  the 
models  and  styles  they  tried  out  for  the  first  time  or  more  formally  than  before.  This  is 
just  our  beginning  and  initial  changes  were  significant. 
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Generally,  all  professors  made  an  effort  to  build  their  2006  courses  incorporating 
constructivism  as  the  process  to  achieve  the  higher  levels  on  Bloom’s  Cognitive  Process 
Dimension  with  students.  Their  goals  were  to  increasingly  engage  students  in 
strengthening  their  metacognitive  skills.  They  reconsidered  their  course  content  structure 
and,  as  a result  of  that  process,  addressed  scaffolding  such  that  the  learning  process  better 
engaged  students  in  building  their  knowledge  and  skills  while  stretching  them  to  achieve 
new  knowledge  and  skill  outcomes.  This  process  began  to  build  a climate  or  lead  them 
into  the  development  of  a “zone  of  proximal  development”  for  their  students.  Remember 
the  2006  semester  was  the  first  attempt  to  begin  building  a learning  environment  different 
from  the  2005  course.  The  professors  had  just  been  introduced  to  many  of  these  concepts, 
but  with  awareness  and  some  understanding,  they  began  to  formally  engage  in  changing 
the  teaching  and  learning  environments  and  climate  for  learning  in  their  classrooms. 
Ultimately,  for  them  to  achieve  deeper  understanding  and  to  document  formal  changes, 
they  will  have  to  engage  in  semesters  of  that  change,  gradually  making  more  and  more 
changes  - each  time  with  deeper  understanding  of  the  theories,  concepts,  and  models. 

This  initiative  was  the  pilot  to  begin  that  process. 

The  research  semester  was  successful  in  implementing  new  learning  models  and 
styles  and  seemingly  to  provide  a wider  range  of  student  learning  style 
opportunities,  appropriate  for  their  level  of  awareness,  knowledge,  skills,  and 
experience.  The  new  teaching  models  and  styles  strengthened  the  possibility  of 
engaging  students  more  actively  (Dale,  1969)  and  at  higher  cognitive  process  levels 
(Bloom,  1956).  The  professors  gained  significantly  in  the  knowledge  about  teaching 
models  and  styles;  they  gained  significant  experience  in  trying  new  ones  in  the  2006 
semester.  Although  there  is  much  more  to  learn,  experience,  research,  and  consider, 
each  professor  in  this  initiative  made  significant  change  using  what  they  learned. 

The  charts  below  present  data  that  indicate  what  they  have  learned  and  what  they 
still  need  to  learn  about  and  be  willing  to  try. 

See  the  Summary  Tables  B.IO  a and  b for  both  an  individual  and  collective  picture 
regarding  professor  responses  to  considering  the  Joyce  et  al.  models. 
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The  following  Map  was  used  to  organize  and  report  teaching  models  (Joyce  et  al,  2004). 
Figures  B.l 1.1-8 


The  Learning  Environment:  Modeis  of  Teaching 


Each  professor  created  a Models  of  Teaching  Map  for  their  2006  experimental  courses. 
See  models  below. 
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CEET  Faculty  Members  Teaching  Model  Maps  2006  Courses 


The  Learning  Environment:  Models  of  Teaching 

student  Learning  Communities 


Student  Teams  of  Expert  Learners 


(Azad,  2007;  based  uponScarborough,  2007  &Teaching  Models  by  Joyce  et  al,  2004) 


C3ra.|=>l-i  EZ>isp>la.v'  ti  i n g 

rvJote  : 

smiley  face  a.ffa.d~ied  to  a model 
iodioa.tes  i_isii~i^  of  tLiat  model. 

- I f *|-ie  smiley  fa oe  a |=> |=»ea rs  at  tl-ie  to |=»  of 

tft^  m o *~1  I i i-ic:l  ioates  usag^  of  tl~iat 

model  as  stated  earlier. 

— If  tt~ie  smiley  fa.oe  apspsea-ts  a.t  tl~ie  dottom  of 

tMe  model  ir~idioates  |;:3artia.l  (_isage  or' 


The  Learning  Environment:  Models  of  Teaching 


student  Learning  Communities 


^ductiv^  /CcWRpK 
^hinkin^  Mtainment 


^^imulatior^  > 

^ ' \Leaming/ 


\ X \Leaming/ 


Student  Teams  of  Expert  Learners 


Coller,  2007;  based  uponScarborough,  2007  &Teaching  Models  by  Joyce  et  al,  2004) 


5 


The  Learning  Environment:  Models  of  Teaching 


student  Learning  Communities 


^^imulatior^  / Social  'v 
N.  y \Learnlna/' 


Student  Teams  of  Expert  Learners 


Gupta,  2007;  based  on  Scarborough,  2007  &Teaching  Models  by  Joyce  et  al,  2004) 


C3  r"s.p3t~i  Icssil  ct  i s pa  I •t^a.cDtn  I t~»Q  m o ct  I s 

l\l  oX&  : 

"T~  l~i  & smiley  To  mocd^l 

incJic^^t^s  usir-iQ  of  mocdol. 

I ~f  m ■ I f <z:  ^a  t <z>f 

mod^l  ■ r~id  i caf 

I f t h~i  o smiloy  f&oo  &|0|oo&r"s  at  t r~i  o looftom  of 
t l~i  o modol  ir-icJio&fos  lo&rtial  uj  o or 

suoooss-  l\/I  ay  oood  f i_J  rt  l~i  o r t\/voal-ci  r-i  q _ 


The  Learning  Environment:  Models  of  Teaching 


lENG  475  (Rahn) -based  onScarborough,  2007  &Teaching  Models  byHo^CS'^al,  2004) 
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The  Learning  Environment:  Modeis  of  Teaching 


Student  Learning  Communities 


/ondirectivV  / personalX 

y^aohmy'  y > 


Student  Teams  of  Expert  Learners 


Moraga,  2007  based  onScarborough,  2007  &Teaching  Models  by  Joyce  et  al,  2004} 


The  Learning  Environment:  Models  of  Teaching 


Tatara,Tech  344,  2007  based  on  Scarborough,  2007  &Teaching  Models  by  Joyce  eTal,  2064) 
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The  Learning  Environment:  Modeis  of  Teaching 


Student  Learning  Communities 
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Student  Teams  of  Expert  Learners 


Motaleb,  2007;  based  on  Scarborough,  2007  &Teaching  Models  by  Joyce  et  al,  2004) 
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Group  Implementation  Summary  of  Individual  Responses 

Jule  Dee  Scarborough,  Ph.D. 


Table  B.  1 1 .a.  1 : Foundation:  Concepts  that  Apply  to  Learning 


Concepts  - 
Chapter  1,  p.3 

Description: 
Professors 
defined  each 
concept/model. 

Meaning  for  Me  and  my  Practice 

Changes  I will  make  based 
upon  my  understanding  of 
this  concept? 

Where  will  these  changes 
“show  up”  in  the  teaching 
and  learning  experiences 
throughout  the  semester? 

2006  Confirmations 
Y=yes,  will  continue 
P=partial,  tried;  will  try  again 
C=did  not  try,  but  committed 
N=did  not,  will  not  later 

Constructivism, 

p.I2 

6 professors  expressed 
understanding  of  the  concept  as 
philosophy,  model  or  strategy.  5 
itemized  ways  to  implement 
constructivism.  There  was  1 NR  and 
1 need  to  improve 

6/7  professors  discussed 
changes  they  could  make. 

One  professor  did  not  respond 
and  had  admitted  in  previous 
column-needed  to  improve. 

Each  professor  identified 
where  or  how  they  would 
implement  changes  in  the 
course  to  engage  student  in 
contructivism 

5 Yes 
2 NR 

Metacognition, 
p.  14 

Professors  discussed  various  ways 
of  tapping  into  metacognition  more 
often  (e.g.  understand  process  of 
learning;  monitoring  own  learning; 
the  role  of  good  assessment,  etc.) 

5 identified  the  changes  they 
could  make;  1 NR. 

5/7  professors  itemized 
ways  to  increase 
metacognitive  engagement 
by  students,  some  very 
specific  examples;  2 NR 

4 Yes 

1 Partial 

2 NR 

Scaffolding, 

p.I4 

Professors  expressed  understanding 
of  the  concept;  some  provided 
examples  of  how  to  implement  or 
how  they  already  implement 
scaffolding. 

3 NR;  1 indicated  scaffolding 
already  in  place;  3 itemized 
how  they  would  implement 
scaffolding. 

3 NR;  1 continually;  2 
provided  specific  examples. 

5 Yes 
2 NR 

Zone  of 
Proximal 
Development, 
p.I6 

(optimal  mismatches 
with  tasks  and  students) 

Professors  expressed  the 
understanding  of  importance  of  this 
concept  and  indicated  that  they 
needed  to  make  this  happen. 

3 NR;  3 described  the  changes 
they  would  make;  1 indicated 
already  in  place. 

5 identified  specific 
examples;  1 yes;  1 not 
possible  to  interpret. 

3 Yes 
2 Partial 
2 NR 

Roles  of  Expert 
Performance, 

p.2 

1 NR;  5 professors  expressed 
general  understanding  of  how  to 
accomplish  in  class;  1 yes 

2 NR;  1 Not  sure;  4 
descriptions  of  how  this  can 
happen. 

3 NR;  1 Not  sure;  4 specific 
examples  of  how  to 
accomplish 

2 NR;  1 No;  4 Yes 
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Table  B.  1 1 .a.2: 1.  Models  of  Teaching  - Information  Processing  Models 


Models 

Description: 
Professors 
defined  each 
concept/model. 

Strengths 

Weaknesses 

How  I can  use  this  model- 
describe 

2006  Confirmations 
Y=yes,  will  continue 
P=partial,  tried;  will  try  again 
C=did  not  try,  but  committed 
N=did  not,  will  not  later 

Inductive 

thinking,  ch.3,  p.4i 
(classification-oriented) 

Professors  expressed  understanding 
of  model. 

3 NR;  4 identified  specific 
weaknesses. 

6/7  identified  specific  ways 
to  implement  model 

4 Yes 
2 Partial 
1 NR 

Concept 

Attainment, 

Ch.4,  p.59 
(includes  concept 
formation) 

3NR/? 

4 professors  expressed 
understanding. 

1 NR;  1 didn’t  see  any  use  for 
model; 

5 specified  weaknesses 

4 expressed  model  as  not 
useful;  3 NR 

6 No 
1 NR 

The  Picture- 
Word  Inductive 
Model 
Ch.5,  p.77 

Although  the  professors  appear  to 
understand  the  model;  they  did  not 
find  it  acceptable  for  use. 

Same 

Same 

1 will  use  to  introduce 
symbols 
4 No 

1 Not  applicable;  1 NR 

Scientific 
Inquiry,  Ch.6, 
p.lOI 
Inquiry 
Training 

3NR;  4 expressed  understanding  and 
results  related. 

*Disappointing  to  have  3 NRs  on 
such  a critical  model  for 
engineering/technology 

2 NR;  5 identified  specific 
weaknesses 

7 identified  specific  uses 

4 Yes 

2 Partial,  still  committed 
1 NR 

Mnemonics, 
Ch.7,  p.131 

(memory  assists) 

5 professors  expressed 
understanding  by  providing 
examples  of  how  to  use-strengths. 
2 NR 

5 itemized  weakness 
2 NR 

2 specified  how  they  would 
use  this  model 
5 felt  not  useful 

1 Partial 
5 No 
1 NR 

Synectics,  Ch.8, 
p.I55  (includes 
metaphoric 
activity) 

4 acknowledge  strengths  well 
2 NR;  1? 

4 specified  weaknesses 
2 NR 
1? 

1 felt  the  model  has 
potential 

2 will  not  use;  3 NR;  1 ? 

2 Partial;  committed  to 
trying 
4 No;  1 NR 

Advance 

Organizers,  Ch.9, 
p.187 

2 NR 

5 felt  positive  about  using  AO, 
specifying  how-strengths 

3 NR 

4 identified  weaknesses 

2 NR 

5 confirmed  model  as 
applicable;  gave  example 

4 Yes 

1 Committed  to  trying 
1 No;  1 NR 
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Table  B.  11. a.3:  II.  M 

[odels  of  Teaching  - Social  Models 

Model 

Description: 
Professors 
defined  each 
concept/mod 
el. 

Strengths 

Weaknesses 

How  I can  use  this 
model  -describe 

2006  Confirmations 
Y=yes,  will  continue 
P=partial,  tried;  will  try 
again 

C=did  not  try,  but 
committed 

N=did  not,  will  not  later 

Partners  in 
Learning,  Ch.lO, 
p.205 

Positive 

Interdependence, 

p.211 

2 NR 

5 specified  strengths 

1 NR 

6 specified  weaknesses 

1 NR 

6 specified  examples 

5 Yes 
2 Partial 

Structured 
Inquiry,  p.221 

1 NR 

6 professors  specified 
strengths 

SNR 

2 identified  weaknesses 

5 identified  where  or 
how  they  would  use 
this  model 
2NR 

3 Yes 

1 Committed  to 
trying 

2 No;  1 NR 

Group 

Investigation 

p.213,14-227 

2 NR 

5 specified  how  to  use  - 
strengths 

2 NR 

5 specified  weaknesses 

3 NR 

4 specified  examples  of 
use 

3 Yes 
2 Partial 
1 No;  1 NR 

Role  Playing, 
Ch.II,  p.229 

2 NR 

5 specified  strengths  or 
uses 

1 NR 

6 specified  weaknesses 

1 would  like  to  try  later 
3 NR 
3 Not 

applicable/practiced/relevant 

6 No;  1 NR 

Jurisprudential 
Inquiry,  Ch.ll, 
p.249 

4 NR 

3 specified  uses  and 
strengths 

3 NR;  1 not  relevant 
3 specified  weaknesses 

2 NR;  1 not  practical 
4 provided  uses  in  class 

1 Yes 

1 committed  to  trying 
4 No;  1 NR 
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Table  B.ll.a.4:  III.  Models  of  Tead 

hing  - Personal  Family 

Model 

Description: 
Professors 
defined  each 
concept/mod 
el. 

Strengths 

Weaknesses 

How  I can  use  this 
model-descrihe 

2006  Confirmations 
Y=yes,  will  continue 
P=partial,  tried;  will  try 
again 

C=did  not  try,  but 
committed 

N=did  not,  will  not  later 

Nondirective 
teaching,  ch.l2, 
p.271 

SNR 

5 specified  strengths 

2 NR 

5 specified  weaknesses 

4 NR 

3 specified  uses,  with 
doubt 

1 Partial 
1 Committed  to 
trying 
SNo 

Enhancing  Self- 
esteem, ch.l3, 
p.283 

3 NR;  1 “Blah” 

3 specified  strengths 

3 NR;  1 “Blah” 

3 specified  weaknesses 

3 NR;  1 “Blah” 

3 provided  examples 

2 Yes 

3 NR;  1 “Blah” 
2 No 

Conceptual 
Development 
Ch.l3,  p.290 

3 NR;  1 not  in  book 
3 specified  strengths 

SNR 

2 specified  weaknesses 

1 specified  examples  of 

use 

SNR 

1 feels  too  difficult  to 
sue 

1 Partial 

2 Committed  to 
trying 

1 No 

3 NR 
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B. 10.2.5:  IV.  Models  of  Teachini 

g - Behavioral  Models 

Model 

Description 
: Professors 
defined 
each 

concept/mo 

del. 

Strengths 

Weaknesses 

How  I can  use  this 
model-descrihe 

2006  Confirmations 
Y=yes,  will  continue 
P=partial,  tried;  will  try 
again 

C=did  not  try,  but 
committed 

N=did  not,  will  not  later 

Mastery 
Learning,  ch.l4, 
p.303 

Programmed 
Schedule,  p.310 
Programmed 
Schedule,  3.11 
(task 

performance 

reinforcement) 

SNR 

4 specified  strengths  or  uses 

2 NR 

2 not  practical 

3 specified  weaknesses 

SNR 

2 will  not  use 

1 Yes 

1 Partial 
3 No 

2 NR 

Direct 
Instruction, 
Ch.l5,  p.313 

1 NR 

6 specified  strengths, 
especially  “efficient” 

1 Nr 

6 specified  weaknesses 

2 NR 

1 will  not  use 
4 indicated  already  in 
use  or  applicable 

5 yes 
1 No 
1 NR 

Simulation, 
Ch.l6,  p.323 
Training  and 
Self-Training 

1 NR 

6 specified  strengths  or  uses 

1 NR 

6 specified  weaknesses 
or  comments 

1 NR 

6 specified  examples  of 
use 

3 Yes 
3 No 
1 NR 

Social  Learning, 
Ch.l4 

(includes  training 
& self-training) 

4 NR;  1 not  in  book 
2 specified  strengths 

4 NR 

3 specified  weaknesses 

1 will  foster  through 
group  work 
SNR 

1 Yes 

1 Partial 

2 No 

3 NR 
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Table  B.ll.b.l:  Individual  Responses 

Jule  Dee  Searborough,  Ph.D. 

Foundation;  Concepts  that  apply  to  Learning  - Models  of  Teaching.  Joyce,  Weil,  Calhoun  (2004) 2006  Confirmations 


Concepts  - 

Description 

Meaning  for  Me  and  my 

Changes  I will  make  based 

Where  will  these  changes  “show 

2006  Confirmations 

Chapter  1,  p.3 

Practice 

upon  my  understanding  of  this 
concept? 

up”  in  the  teaching  and  learning 
experiences  throughout  the 

Y=yes,  will  continue 
P=Partial,  tried;will  try  again 
C=did  not  try,  but  committed 

semester? 

N=did  not,  will  not  later 

Constructivism, 

Each 

“Teaching  by  direct  method 

“Even  though  the  current 

“These  changes  will  show  up 

Yes 

p.l2 

professor 

(i.e.  teaching  the  content)  as 

course  has  both  methods,  it  is 

through  lectures  as  well  as 

provided  a 

well  as  discovery  method 

mostly  direct  method. 

performance  tasks  assigned 

description  or 

(how  to  learn  content)  so 

Proposed  change  will  bring  in 

throughout  semesters.” 

explanation  of 

that  student  can  construct 

more  discovery  method  to 

all  concepts  or 

knowledge  appropriately.” 

create  a balance  of  these  two 

models  listed 
on  this 

methods.” 

worksheet 

“Needs  to  be  improved.” 

NR 

“Tinker  Toys” 

Yes 

from  their 
study  of  the 

“Although  I have  to 

“The  topics  seen  in  my  class 

“No  sure  after  reading  all 

Yes 

hook. 

transmit  knowledge  to  my 

require  students  to  create 

chapters.  I guess  the  topic  of 

students,  I have  also  to  look 

knowledge.  I already  make 

problem  formulation  could  be 

for  the  way  how  they  will 

students  work  in  groups,  but  I 

the  more  appropriate.” 

build  up  more  knowledge.” 

have  to  plan  better  activities.” 

“Through  the  use  of  discussion 

NR 

“Essential  to  the 

“I  will  incorporate  more 

groups,  case  studies,  and 

transferring  of  usable 

assessments  and  lecture  time 

structured  performance  tasks.” 

knowledge  to  the  students. 

to  active  learning  and  higher 

and  to  encouraging  the 
development  of  knowledge.” 

level  of  Bloom’s.” 

“Throughout  the  duration  of  the 

NR 

“Burden  of  gathering 

“-facilitate  various  modes  of 

course.” 

knowledge  lies  with 

knowledge  gathering 

students.  As  a teacher,  I am 

environment  for  the  students.” 

going  to  provide  the  means 
of  faciUtating  that  process.” 

“organized  group  project  in 

“performance  task  #3  and  final 
exam.” 

Yes 

“use  groups  to  help  students 

performance  task  3” 

learn.” 

“give  them  projects  through 

“project” 

Yes 

NR 

which  they  can  create 
knowledge.” 
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Metacognition, 

“Student  is  encouraged  to 

“Explain  to  students  not 

“While  solving  problems,  this 

Yes 

p.  14 

not  only  to  master  the 

only  the  procedure  to  solve 

approach  will  be  followed,  i.e. 

subject  but  also  understand 

a particular  problem  but 

problem  statement  variables  to  be 

the  process  of  learning  and 

also  how  can  it  be 

solved  (generalization)  as  well  as  the 

applying  it  for  other 

generalized,  e.g. 

procedure  (content  specific).” 

situations.” 

converting  a physical 
statement  into 
mathematical  problem. 
Provide  immediate 
feedback.” 

“Need  to  improve.” 

NR 

“Many  examples  of  calculating 

Yes 

Students  get  a week 

movement  and  different  ways  of 

later;  not  sure  if  the  close 
the  feedback  loop.” 

doing  it.” 

“My  students  should  be 

“Not  sure  about  it.  I think 

“Not  sure.” 

Partial 

provided  of  way  to  monitor 

the  assessment  process  is  a 

their  learning  progress.” 

valid  way  to  provide 
students  with  that 
mechanism.” 

“this  is  the  key  to  the 

“I  will  include  activities 

“Through  the  use  of  discussion 

NR 

concept  of  hfelong 

that  encourage  them  to 

groups,  case  studies,  brain  teaser 

learning.” 

think  about  how  they  are 
thinking.” 

exercises,  and  performance  tasks.” 

“Students  should  develop 

“Through  the  duration  of  the 

NR 

control  over  their  learning 

“-performance  tests, 

course.” 

processes  rather  than 

groups  discussions,  self- 

passively  reacting  to  the 

reflection” 

environment.” 

“use  test  item  bank  to 

“Performance  tasks  1-3  and  midterm 

Yes 

“must  use  assessment  over 

make  sure  all  topics  are 

and  final  examinations.” 

wide  range;  good 
assessments  must  be 
employed.” 

assessed  in  exams.” 

NR 

NR 

NR 

Yes 
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Scaffolding, 

p.l4 

“Make  the  student  more 
independent  as  they 
progress  through  the 
course.” 

“Initial  assignments  will 
have  more  information  but 
later  assignments  will  have 
less  information.” 

“Assignments  or  laboratory  exercises 
(such  as  Lab  1 and  2),  most  of  the 
information  will  be  provided  in 
detail,  but  for  laboratory  3,  less 
information  will  be  provided.” 

Yes 

“I  only  have  a chance  to  do 
this  when  students  come  to 
see  me  - rare.” 

NR 

“In  class  activities” 

Yes 

“I  should  provide  these 
ways  of  metacognitive 
control.” 

“The  idea  of  reciprocal 
teaching  is  something  I 
would  like  to  try.” 

“I  think  problem  formulation  is  the 
open  place  where  I can  incorporate 
this.” 

Yes 

“This  is  necessary  if  we 
want  them  to  understand 
how  to  think.” 

“I  will  try  this.  I do  not 
yet  know  exactly  how  this 
will  unfold.” 

NR 

NR 

“Providing  students  with 
support  while  introducing  a 
concept  or  topic.  The  level 
of  support  will  he  reduced 
gradually  as  the  students 
master  the  concept.” 

NR 

NR 

NR 

“already  being  executed  in 
classroom  instruction.” 

“already  in  place” 

“continually” 

Yes 

NR 

NR 

NR 

Yes 
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Zone  of 

“Some  mismatch  so  that  the 

“While  explaining  a 

“For  performance  tasks  students  will 

Yes 

Proximal 

student  is  not  too 

problem,  initially  not 

initially  work  individually  and  since 

Development, 

comfortable  and  yet  not 

much  explanation  will  be 

some  are  very  rigid  (stage  I of 

p.l6 

overwhelming  so  that  the 

given  and  students  will  be 

Optimal  Environment)  to  adaptable 

(optimal 

student  is  not  frustrated.” 

asked  to  participate  to 

(state  IV),  they  may  face  different 

mismatches 

complete  the  problem  (so 

challenges.  However,  when  they 

with  tasks 

that  they  are  challenged). 

work  to  arrive  at  best  solution,  they 

given  to 

However  if  they  cannot 

learn  to  be  adaptable.” 

students) 

respond  then  instead  of 
completely  solving  the 
problem,  more  hint  will  be 
given  (so  that  they  are  not 
frustrated)  to  make  them 
think  and  learn.” 

“Important!  I don’t  do  this 
now.  Difficult  to  keep  70 
students  in  zone.  All  have 

NR 

“Perhaps  (HW?)that  progress.” 

Partial 

different  paces.” 

Yes 

“I  should  find  the  optimala 

“While  they  work  in 

“I  think  in  problem  formulation. 

environment  for  my 

groups,  I should  observe 

students  to  work.” 

them.  I don’t  know  how  to 
kind  of  ‘measure’  this.” 

“This  facilitates  the 

“These  will  be  enumerated 

“This  will  be  demonstrated  through 

NR 

application  and  synthesis  of 

in  the  performance  task 

completion  of  the  new  performance 

knowledge  needed  to  reach 

goals  and  processes  they 

task.” 

the  higher  levels  of 

will  need  to  engage  in  to 

Bloom’s.” 

complete  the  tasks.” 

“-performance  tests;  open-ended 

NR 

“To  provide  students  with 
tasks  and  problems  that 
challenges  them.” 

NR 

items” 

“buld  from  simple  concepts 

“already  in  place.” 

“continually,  lectures  and  student 

Yes 

to  more  complex.” 

learning  assignments.” 

“force  them  to  5-6  Bloom’s  level” 

NR 

J (assumed  yes) 

Partial 
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Roles  of  Expert 
Performance, 

p.20 

“Have  a broad  vision  in 
terms  of  [course  content] 
teaching  and  implement  in 
appropriate  level.” 

“Keep  focus  on  ultimate 
goal  of  designing  for 
[course  content]  in  various 
situations  and  ensure  that 
the  classes  are  eventually 
geared  towards  that  goal.” 

“Assignments  are  built  around  the 
central  theme  of  designing  for 
vibration  and  let  students  appreciate 
that.” 

Yes 

“Model  good  problem 
solving  at  board,  but  might 
do  a better  job  of  breaking 
it  down.” 

NR 

NR 

Yes 

“This  is  what  I try  to 
accomplish  in  my  class.” 

“No  sure.” 

“Not  sure.” 

No 

“Essential  to  active  learning 
and  higher  levels  of 
Bloom’s.” 

“The  students  will  need  to 
engage  in  this  to  complete 
their  tasks.” 

“This  will  be  demonstrated  and 
assessed  through  the  use  of  the  new 
performance  tasks.” 

NR 

“to  challenge  the  student 
with  higher  level 
performance  at  the  very 
beginning  instead  of 
allowing  them  to  proceed 
progressively.” 

“There  is  a possibiloity  to 
introduce  this  concept.” 

NR 

NR 

“address  projects  to  levels 
of  5 and  6 of  Bloom’s, 
mostly  6;  strive  for  high 
goals.” 

“give  assessments  that 
challenge  students  at 
Bloom’s  levels  5 and  6.” 

“Performance  tasks  1-3  at  higher 
levels  of  Bloom’s.” 

Yes 

NR 

NR 

NR 

Yes 
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Table  B. 10.3.2: 1.  Models  of  Teaching  - Information  Processing  Models 


Models 

Description 

Strengths 

Weaknesses 

How  I can  use  this  model-describe 

2006  Confirmations 
Y=yes,  will  continue 
P=Partial,  tried;will  try  again 
C=did  not  try,  but  committed 
N=did  not,  will  not  later 

Inductive 

Each 

Students  may  learn  how 

May  not  follow  text  book 

Make  students  classify  a [field-specific] 

Yes 

thinking,  Ch.3, 

professor 

and  why  various  topics  are 

and  thus  text  book  may 

problem  in  terms  of  various  categories 

p.41 

provided  a 

organized  and  developed 

not  be  of  as  much  help. 

[name  and  list],  and  approach  solution 

(classification- 

description 

and  then  become  able  to 

accordingly 

oriented) 

or 

generalize  it.  Helps 

explanation 
of  all 

students  to  see  the  pattern. 

concepts  or 

Help  students 

NR 

Categorize  [gave  course  specific 

Yes 

models  listed 
on  this 

conceptualize;  see  patterns 

examples] 

worksheet 

It  allows  students  to 

The  support  systems 

I think  in  [course  specific  example]  I 

Partial 

from  their 

engage  in  inquiring  and 

require  a wide  source  of 

could  use  this  model.  I should  have  to 

study  of  the 

logical  thinking 

unclassified  data. 

design  an  example  where  they  can  play 

book 

with  it. 

This  model  is  very 

Very  time  consuming. 

Through  performance  tasks-they  must 

NR 

effective. 

take  their  information  and  assimilate  it, 
organize  it,  and  decide  how  to  use  it. 

It  appears  to  be  useful  for 

NR 

-classification  of  [course  specific] 

Yes 

classification  purpose 

problems  in  terms  of  design  approaches 

Students  are  natural 

Students  must  practice  to 

Classify  items;  yes,  good  for 

Partial 

conceptualizers.  Increases 

think  inductively; 

engineering  [and  technology]  classes. 

range  of  perception  from 

textbooks  may  not  be 

which  students  view 
information. 

oriented  to  method. 

NR 

NR 

Appropriate 

Yes 
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Concept 

Try  to  understand  the 

May  take  too  long  and 

NR 

No 

Attainment, 

underlying  common  theme 

may  not  be  suitable  for 

Ch.4,  p.59 
(includes 

from  sets  of  data 

particular  class 

concept 

9 

I don’t  see  any  use  for  this 

NR 

No 

formation) 

in  my  class 

Good  to  recognize 
common  patterns 

It  is  for  simple  concepts. 

I don’t  think  I can  use  this  model. 

No 

Students  must  figure  out 
the  attributes  of  a category 
by  comparing  and 
contrasting  examples  that 
contain  the  attributes  of 
the  concept  with  examples 
that  do  not. 

Very  time  consuming. 

I will  not  use  this  technique. 

NR 

NR 

NR 

NR 

No 

Works  best  with  loosely 

Most  textbooks  are  not 

Not  appropriate 

No 

defined  attributes 

suitable  for  concept 
learning  and  toher  sources 
of  information  must  be 
found. 

NR 

Too  long 

No  use 

No 
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The  Picture- 

NOT  in  the  book  edition  I 

No 

Word 

had 

Inductive 

Model 

NR 

Takes  time 

First  week  performance 

No 

Ch.5,  p.77 

NR 

NR 

NR 

Not  applicable 

Aids  in  making 

Mainly  used  for  language 

I will  not  use  this  technique 

NR 

connections  between 

verbal  and  visual  concepts 

NR 

NR 

NR 

To  introduce  symbols 

Leads  students  to  more 

Control  of  process  is  with 

Probably  not  useful 

No 

and  more  complex  tasks; 

instructor,  not  the  student, 

provides  multidimensional 

mostly  suited  for  lower 

curriculum 

grade  students 

NR 

NR 

NR 

No 
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Scientific 

Engineering,  similar  to 

It  is  complex  and 

Home  works  or  laboratory  assignments 

Yes 

Inquiry,  Ch.6, 

science,  is  not  historical 

elaborate 

can  be  formulated  based  on  this. 

p.lOl 

fact.  Inquiry  including 

Used  in  performance  assessments 

Inquiry 

experimentation  is  needed 

Training 

to  find  the  fact  and  that 
fact  may  change  over 
period  of  time  as 
knowledge  progresses 

NR 

Takes  time 

First  wee  performance 

Yes 

Provides  high  levels  on 

It  is  complex 

I will  use  this  method  in  one  of  the 

Partial;  still  committed 

Bloom’s  [Taxonomy] 

performance  tasks;  students  will  be  able 
to  learn  from  that  experience. 

Necessary  to  be  successful 

Very  time  consuming 

This  will  be  accomplished  through  the 

NR 

in  any  area  of  engineering 

use  of  the  new  performance  tasks 

NR 

NR 

-[course  specific]  design  problems  with 
practical  appUcations 
-investigation  of  a given  problem  within 
a [course  specific]  system 

Yes 

Gradually  leads  students 

Data  may  not  be 

Good,  useful  for  engineering  classes  in 

Partial 

to  more  advanced  topics. 

interpreted  properly  to 

group  setting  for  performance  tasks. 

Brings  students  into 

develop  hypothesis. 

scientific  process. 

Instructor  must  have 
great  in-depth  knowledge, 
topic. 

Yes 

NR 

NR 

Use  in  project 
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Mnemonics, 

Students  realize  that  they 

Not  so  useful  for  topics 

Very  little  use.  I can  think  of  using  it  in 

No 

Ch.7,  p.l31 

can  control  and  modify 

where  there  is  less  need 

[particular  course]  where  [example  of 

(memory 

their  own  mental  activities. 

for  memorization 

discipline-specific  use]. 

assists) 

Also,  creativity  is 
nourished  through 
improvement  of  imaging 
capacity 

Only  see  it  valid  for 
deflnitions  such  as 
[formula  discipline 
specific] 

Same 

Same 

No 

Applicable  to  simple 

Less  applicable  to 

I don’t  think  I can  use  this  model. 

No 

concepts 

engineering 

Ties  pattern  to  concept 

Better  for  lower  level 
subject  matter 

I will  not  use  this  technique 

NR 

NR 

NR 

-to  remember  names  and  numbers  of 
various  parts  that  are  used  within  the 
system. 

Partial 

Students  become  more 

Attention  must  first  be 

Not  appropriate 

No 

effective  at  memorizing; 

given  to  what  is  to  be 

empowers  students 
intellectually 

learned 

NR 

Not  useful  in  for  college 

No 

NR 
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Synectics, 

Occasionally  may  be  useful 

Difficult  to  come  up  with 

NR 

No 

Ch.8,  p.l55 

to  understand  the 

examples  which  have 

(includes 

underlying  idea  (example); 

completely  different 

metaphoric 

may  permit  creativity. 

appearance  but  somehow 

activity) 

explains  the  fundamental 
concept 

9 

9 

9 

No 

Stimulate  imagination 

Requires  elaboration 

This  is  something  I would  like  to  try. 
The  book  doesn’t  show  how  in  case  of 
science.  Seems  to  have  a lot  of 
potential. 

Committed  to  trying 

Illustrates  how  things  are 

Very  time  consuming  and 

I will  not  use  this  technique 

NR 

connected,  and  eventually, 
if  the  path  is  long  enough, 
they  may  no  longer  be 
related 

very  abstract 

NR 

NR 

NR 

No 

Enhances  creativity  of 

High-achieving  students 

No  appropriate,  but  used  in  lectures  to 

Partial 

individuals  and  groups; 

reluctant  to  participate; 

connect  what  students  learned 

builds  feeling  of 

students  must  be  exposed 

elsewhere. 

community;  combines 
easily  with  other  models 

to  process  repeatedly. 

NR 

NR 

NR 

No 
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Advance 
Organizers, 
Ch.9,  p.l87 

Easy  for  students  and 
teachers  to  build  the 
concept  around  this  theme 

Not  as  effective  if  concepts 
are  completely  new 

This  is  applicable  in  my  courses  as 
students  see  the  big  picture  and  also 
how  small  pieces  fit  in  that  big  picture 

Yes 

Efficient 

NR 

This  is  what  I do 

Yes 

It  is  easy  to  implement  and 
allows  to  present 
conceptual  structure 

It  can  easily  drive  to 
passive  learning  if  not  well 
planned. 

It  is  definitely  applicable. 

Committed  to  trying 

Good  for  tying  together 
multiple  concepts  needed 
for  the  solutions  to 
complex  problems. 

Time  needed  to  set  this  up 

This  will  definitely  be  used  in 
lectures/scenarios  to  the  multiple 
concepts. 

NR 

NR 

NR 

NR 

No 

Good  for  systematic 
education  of  students  in  a 
key  idea  of  a field. 

Content  must  be  well 
prepared  and  organized; 
student  must  be  prepared 
to  learn. 

Already  in  use. 

Yes 

NR 

NR 

NR 

Yes 
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Table  B.10.3.3:  II.  Models  of  Teaching  - Social  Models 


Model 

Description 

Strengths 

Weaknesses 

How  I can  use  this  model-describe 

2006  Confirmations 
Y=yes,  will  continue 
P=Partial,  tried;will  try  again 
C=did  not  try,  but  committed 
N=did  not,  will  not  later 

Ch.lO,  Partners 
in  Learning, p.205 

Positive 

Helps  students  to  learn 

All  students  must 

Laboratory  assignments  and 

Partial 

Interdependence, 

cooperatively.  Each 

participate  equally.  Group 

performance  assessment 

p.211 

student  has  few  tasks  to 

should  be  small  for 

master  and  thus  can  learn 

appropriate  testing 

“training  effects” 

those  well 

Individuals  in  group  are 

Tasks  must  be  simple,  e.g. 

NR 

Yes 

rewarded  when  whole 

spelling  things  that  can  be 

group  achieves 

trained 

Help  students  learn 

Students  must  participate 

Project  works. 

Yes 

collaboration. 

equally.  Current  testing 
discourages  collaboration 
learning. 

Stresses  cooperation  and 

Difficult  to  assess  how 

Through  cooperative  learning  and 

NR 

the  fact  that  all  have  an 

equally  all  have 

group  activities. 

important  role  to  play. 

participated. 

-performance  tasks  can  be  designed  as 

Yes 

NR 

NR 

group  investigation  (Partners  in 
Learning  comment) 

-some  of  the  activities  can  be 
incorporated  during  group  meetings 

Partial 

Helps  students  learn  to 

All  students  must 

Performance  Task  #3 

Yes 

work  cooperatively. 

participate  equally; 

Individuals  are  left  with 

groups  must  be  smaller; 

few  tasks  to  master. 

current  testing 
discourages  cooperative 
learning  [K-I2  only-not 
higher  ed] 

NR 

Groups  must  be  small;  current 
tests  discourage  group  learning 

Project 

Yes 
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Structured 
Inquiry,  p.221 

Students  learn  effectively 
by  this  method  because  it 
is  an  active  learning 

NR 

Used  in  homework  and  performance 
assessments 

Yes 

Motivating 

NR 

All  3 performance  tasks 

Yes 

Students  share  ideas  and 
viewpoints. 

NR 

Completely  applicable. 

No 

Excellent,  especially  in  the 
use  of  case  studies  at  the 
higher  education  level. 

Need  a good  problem. 

I already  do  this-it  will  be  included  in 
the  solving  of  the  case  studies. 

NR 

NR 

NR 

NR 

No 

Conflicting  viewpoints 
interest  student;  students 
learn  to  reason  and 
negotiate  and  develop 
capacity  for  reflection. 

Students  must  be  both 
observers  and 
participants;  it  requires 
firsthand  activity  in  a real 
situation  to  collect  data. 

All  performance  tasks 

Yes 

Students  see  conflicting 
points  of  view  [and]  make 
them 

NR 

NR 

Committed  to  trying  later 
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Group 
Investigation 
p.213, 14-227 

Proven  effective  way  of 
learning 

Possibility  that  not  all 
students  are  equally 
participating 

Used  in  performance  assessments 

Partial 

Proven  effective 

Have  to  manage  group 
issues. 

Performance  tasks;  inductive  activities 
in  class 

Yes 

Students  are  taught 
democratic  process. 

Instructor  must  have  high 
level  inter-process  skills. 

It  has  potential. 

Partial 

Group  work  is  beneficial 
in  many  ways — here  the 
method  of  inquiry  is 
particularly  important. 

Constrained  by  structure 
in  the  group. 

Some  of  this  will  be  accomplished 
through  group  performance  tasks. 

NR 

NR 

NR 

NR 

No 

Teaches  democratic 
process;  enhances  social 
education  of  students. 

Instructor  must  have  high 
level  of  personal  and 
instructional  skills; 
process  is  slow  and 
cumbersome;  have  to 
manage  group  conflict. 

NR 

Yes 

NR 

NR 

NR 

Yes 

28 


Role  Playing, 

Understanding  empathy, 

Lot  of  classroom  time 

NR 

No 

Ch.ll,  p.229 

conflict  resolution,  how  to 
listen  to  other  students 

needs  to  be  devoted 

NR 

Not  Relevant 

NR 

No 

Great  value  for  developing 

Requires  a great  deal  of 

Not  applicable. 

No 

empathy. 

time  in  classroom. 

Great  to  tie  together 

Very  time  consuming  and 

I would  like  to  try  this  in  the  future. 

NR 

interdiscipUnary 

get  these  types  of  activities 

concepts — such  as 

prepared  for  technical 

language  and  math. 

classes  at  the  higher 
education  level. 

NR 

NR 

NR 

No 

Gives  greater 

One  problem  is  that  role 

Not  practiced 

No 

understanding  and 

playing  requires  in-depth 

empathy;  allows  for 

treatment  that  needs  a 

conflict  resolution 

great  deal  of  classroom 

uniquely  encouraging 
listening  to  other  students. 

time. 

Gives  better 

Requires  a great  deal  of 

Not  relevant 

No 

understanding 

time  in  class 
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Jurisprudential 
Inquiry,  Ch.ll, 
p.249 

May  help  understand 
application  in  the  context 
of  possible  litigation.  Also 
issue  of  ethics  can  be 
discussed  this  way. 

Need  to  find  appropriate 
and  relevant  cases 

If  an  impact  problem  has  to  be 
addressed  (course  apphcable  example 
that  works  to  explain  the  application) 

Yes 

NR 

Interesting,  but  not 
relevant 

NR 

No 

NR 

NR 

To  incorporate  ethics. 

No 

Approaches  problem 
solving  from  the 
argumentative  side — one 
looks  at  a problem  and  a 
solution  from  more  than 
one  angle. 

Very  specific  in  form. 
However,  I see  the  use  for 
the  multiple  side  concept. 
Perhaps  this  can  be  done 
without  a “court  case”. 

I would  like  to  try  this  in  the  future 
without  the  constraints  of  a “course 
case”. 

NR 

NR 

NR 

NR 

No 

Helps  people  rethink  their 
position  of  legal,  ethical, 
and  social  issues;  gives 
tools  for  debates. 

One  of  the  difficult  tasks  is 
for  the  instructor  to 
integrate  case  details  into 
a larger  social  issue. 

Not  practical;  linked  to  ethics. 

No 

NR 

NR 

Ethics 

Committed  to  trying  later 
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Table  B. 10.3.4:  III.  Models  of  Teaching  - Personal  Family 


Model 

Description 

Strengths 

Weaknesses 

How  I can  use  this  model-describe 

2006  Confirmations 
Y=yes,  will  continue 
P=Partial,  tried;will  try  again 
C=did  not  try,  but  committed 
N=did  not,  will  not  later 

Nondirective 

Greater  personal 

Cannot  be  planned 

If  needed,  use  on  a case  by  case  basis. 

Did  not  try,  but  committed 

teaching,  ch.l2, 

interaction  for  individual 

beforehand. 

to  try  later 

p.271 

growth  of  students. 

Time  consuming.  Also 

Apparently  favored  by 

question  is  how  much  role 

African  American 

of  counselor  should  be 

students 

played  by  instructor 

NR 

More  of  a one-on-one 
thing  as  situations  arise, 
rather  than  something 
that  is  planned 

NR 

No 

NR 

NR 

NR 

No 

Students  become  engaged. 

Time  allotment. 

This  can  be  done  through  group  work 
and  discussion  sessions. 

NR 

NR 

NR 

NR 

No 

Students’  consciousnesses 

Success  depends  on 

Probably  not  practical,  except  to 

No 

are  raised,  helping  to 

nurturing;  may  result  in 

respect  all  questions  from  the  students. 

clarify  their  ideas;  student 

conflicting  roles  for  the 

is  free  of  pressure  or 

instructor-disciplinarian; 

coercion;  student  is  helped 

replace  friend,  instructor. 

to  grow  as  a person 

etc.?  Very  time 
consuming. 

Students  free  of  pressure 

May  result  in  conflicting 
rules  of  instruction 

More  advising  issues 

Partial 
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Enhancing 

Not  cut  down  the  students 

Need  to  be  administered 

I always  tell  in  class  that  no  question  is 

Yes 

Self-esteem, 

carefully 

stupid  question 

ch.l3,  p.283 

BLAH 

BLAH 

BLAH 

BLAH 

NR 

NR 

NR 

NR 

Relates  ability  to  function 

Not  easy  to  accomplish  or 

I will  strive  to  relate  the  idea  of  lifelong 

NR 

in  the  outside  world  to 

measure.  Perhaps  the  use 

learning.  “They  can  do  it.” 

how  they  are  approached 

of  learning  inventories  can 

inside  the  classroom. 

aid  in  the  instructor’s 
abilities  to  help  with  this 
task. 

NR 

NR 

NR 

No 

Students  gain  strength  to 

Students  must  be  more 

Probably  not  practical,  except  to 

No 

learn  on  their  own;  more 

proactive  to  benefit  and 

respect  all  questions  from  the  students. 

skills  are  developed  and 

draw  knowledge  from 

skill  set  widens  to  be  able 

environment.  Passive 

to  master  even  greater 

students  will  be  by-passed 

skills. 

by  this  learning  process. 

NR 

NR 

Encourage  students  to  ask  questions 

Yes 
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Conceptual 

Not  in  edition  of  book  I 

NR 

NR 

Partial 

Development 

have 

Ch.l3,  p.290 

NR 

NR 

NR 

NR 

NR 

NR 

NR 

NR 

Gets  students  to  “think 

Not  easy  to  accomplish  or 

I will  try  to  get  them  to  think  outside 

NR 

outside  the  box”,  or  their 

measure. 

the  box  and  incorporate  their  new 

comfort  zone. 

knowledge.  I believe  that  performance 
tasks  and  discussion  sessions  can  play  a 
vital  role  in  this. 

NR 

NR 

NR 

Committed  to  trying 

Students  grow  and 

Less  developed  students 

Difficult  to  use  in  engineering  [and 

No 

develop  rich  orientations 

will  spend  more  time 

technology]  classes. 

for  growth.  New 

trying  to  survive  in  new 

experiences  are  tolerated 

environment  rather  than 

and  bring  about  new  ideas 

learning  from  it.  Need  to 

and  thoughts,  rather  than 

balance  areas  of 

the  existing  state. 

knowledge. 

New  experience  is 
tolerated 

NR 

NR 

Committed  to  trying  later 
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Table  B. 10.3.5:  IV,  Models  of  Teaching  - Behavioral  Models 


Model 

Description 

Strengths 

Weaknesses 

How  I can  use  this  model-describe 

2006  Confirmations 
Y=yes,  will  continue 
P=Partial,  tried;will  try  again 
C=did  not  try,  but  committed 
N=did  not,  will  not  later 

Mastery 

Pace  appropriate  for 

Progress  of  class  may  be 

NR 

No 

Learning, 

individual  learning  style 

hindered  due  to  some 

ch.l4,  p.303 

and  pace 

individual  student’s 

Programmed 
Schedule,  p.310 

inability  to  progress 

Programmed 
Schedule,  3.11 

Lets  students  work  at  a 

Impractical  for  [my 

NR 

Partial 

(task 

pace  which  is  appropriate 

course] 

performance 

reinforcement) 

for  his/her  learning  style 

NR 

NR 

NR 

NR 

Students  demonstrate 

Not  practical  on  a large 

I will  not  use  this  technique. 

NR 

proficiency  before  they 
move  on,  and  fundamental 
building  blocks  are  not 
passed  by.  They  do  not 
get  “left  behind”. 

scare  for  larger  classes. 

NR 

NR 

NR 

No 

Instruction  in 

Students  require  different 

Not  practical;  can’t  keep  up  all 

No 

encouragement  role; 

of  times  to  master 

students. 

positive  effect  on  students; 

material;  only  modest 

students  work  at  different 

increase  in  student 

paces  to  learn. 

learning;  no  improvement 
on  tests. 

NR 

Progress  of  class  is  slower 

NR 

Yes 
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Direct 

Efficient,  fast,  has  been 

Limited  participation 

It  is  used  already 

Yes 

Instruction, 

used  traditionally 

from  students  (less 

Ch.15,  p.313 

encouraging) 

Efficient 

Not  engaging 

NR 

Yes 

Efficient  and  fast 

It  might  not  be  engaging 

Applicable 

Yes 

Good  for  lower  level 

Not  practical  for  high  level 

I will  not  use  this  technique. 

NR 

subject  matter. 

material. 

NR 

NR 

NR 

No 

Solid  method;  good 

Limited  participation 

Already  widely  used  in  course. 

Yes 

history  of  getting  results 

from  students;  students  do 
not  answer  or  question. 

Efficient 

Less  engaging 

What  we  do 

Yes 

Simulation, 

Use  software  effectively 

Need  to  be  careful  if  the 

Use  software  such  as  Matlab 

Yes 

Ch.l6,  p.323 

for  simulation 

software  is  good  or 

Training  and 
Self-Training 

providing  correct  idea 

Authentic;  learn  by  doing 

More  natural  to  do  in 

[Course  specific  example]  is  a 

Yes 

some  classes  than  others 

simulation  of  sorts 

Students  can  develop  skills 
equally 

Software  package 

It  has  been  used  in  engineering 

Yes 

Connects  concepts  to 

Time  consuming 

I already  implement  simulation 

NR 

actually  “doing”  and  the 
real  world 

techniques. 

NR 

NR 

NR 

No 

Students  play  roles  of  real 

Most  models  depend  on 

Not  useful  for  course 

No 

people  engaged  in  real  life 

good  software  packages; 

actions 

difficult  to  blend  into 
curriculum. 

Can  see  for  themselves 

Depends  on  software  quality 

Labs 

No 
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Social 

Not  in  my  book 

NR 

Learning, 

Ch.l4 

Coop  Learning 

NR 

NR 

NR 

Yes 

(includes 
training  & self- 
training) 

NR 

NR 

NR 

NR 

This  goes  hand-in-hand 

Not  easy  with  certain  types 

I will  foster  this  through  group 

NR 

with  simulation 

of  personalities 

discussion  sessions. 

NR 

NR 

NR 

No 

Nearly  all  students  can 

Must  manage  class  as  to 

NR 

Partial 

master  tasks  and  achieve  a 

reorganize  curriculum  to 

given  set  of  objectives. 

give  enough  time  for  each 
student  to  learn. 

NR 

Time  consuming  because 
you  have  to  give 

NR 

No 
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(Copied  from  GAPS  Analysis  - Teaching  Models  Data  Embedded;  See  pages  43-47) 
GAPS  Analysis  Summary  (Fall  2005  and  Fall  2006) 

Jule  Dee  Scarborough,  Ph.D. 

Student  Learning  Statements  (Outcomes) 

In  the  initial  analysis  of  the  Fall  2005  courses  - where  we  began,  professors  used  their 
existing  course  syllabi.  Although  as  a college,  we  had  improved  our  student  learning 
statements  during  the  accreditation  process,  they  remained  rather  unorganized  and  weak 
in  content  and  appropriate  expression.  The  learning  statements  were  expressed  in  mixed 
modes  across  syllabi.  Some  learning  statements  were  written  as  course  objectives;  others 
were  written  as  student  learning  objectives;  yet  others  were  written  as  more  outcome- 
oriented  statements.  However,  in  generalizing,  many  and  sometimes  most  of  the  student 
learning  statement  formats  across  syllabi  were  not  active,  clear,  measurable,  or  clearly 
outcomes-oriented,  where  the  professor  and  student  could  ascertain  exactly  what  was 
expected  and  would  be  measured,  and/or  determine  the  culminating  grade.  Three 
professors  expressed  the  learning  statements  in  a way  where  students  could  see  that  there 
was  a relationship  between  student  learning  outcomes  and  the  ABET  or  NAIT  outcomes, 
but  if  the  ABET  or  NAIT  outcomes  were  identified  by  a letter  and  not  stated,  then  the 
relationship  was  not  clear,  nor  were  students  about  to  review  the  accreditation  outcomes 
for  their  own  information.  Two  professors  expressed  the  statements  more  clearly,  with 
written  statements  for  both  the  national  standards  and  the  learning  outcomes  for  the 
course.  The  other  five  professors  did  not  show  the  national  statements  in  narrative  but 
rather  identified  them  by  letter  or  number,  regarding  the  level  of  coverage  and  depth  of 
relationship.  This  had  little  meaning  for  students  and  did  not  make  it  easy  for  the 
professors  to  clearly  be  assured  of  direct  links  and  relationships.  Generally,  the 
statements  did  link  to  the  ABET  or  NAIT  standards  or  outcomes,  but  often  not  clearly  or 
strongly.  It  would  have  been  difficult  to  determine  a direct  link,  especially  in  light  of  the 
student  learning  assessments  being  used  for  the  2005  course.  Therefore,  we  examined  the 
2005  syllabi  and  course  content  related  to  the  standards  as  well  as  we  could,  with  the 
understanding  that  the  student  learning  outcomes  to  be  redeveloped  would  better  and 
more  clearly  and  directly  link  to  the  national  standards  and  assessments  - a two-way  link 
revealing  the  critical  knowledge  and  skill  connections. 

Below  are  two  charts  that  broadly  identify  the  standards  addressed  in  the  Fall  2005 
courses,  according  to  the  content  and  syllabus  analysis  by  each  professor  of  his/her 
course.  The  data  are  presented  (in  black)  as  collapsed  across  either  all  engineering 
professors  or  engineering  technology  professors  as  a broad  viewpoint.  The  Fall  2006 
courses  are  presented  in  red,  and  although  there  are  minor  differences  in  the  number  of 
standards  addressed,  there  is  a great  and  very  significant  difference  in  the  quality  of  the 
learning  statements  and  their  direct  links  to  both  the  national  standards  and  the  learning 
assessments.  The  tables  also  reflect  the  number  of  learning  outcomes  for  each  standard  by 
professor,  2005-2006  when  possible.  For  the  2006  courses,  the  professors  not  only  have 
improved  wording  and  expression,  but  the  knowledge  and  skill  connections  are  much 
stronger;  in  addition,  the  outcome  statements  are  improved  because  they  are  broken  out 
into  primary,  second,  and  third  level  statements.  The  quality  is  improved  not  only 
because  of  better  wording,  but  also  because  they  now  better  understand  the  difference 
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between  complex  statements,  where  there  is  a cluster  of  outcomes  inherent  to  a single 
primary  outcome  statement.  Thus  the  course  content  or  the  knowledge  and  skills  to  be 
taught  became  more  obvious  in  the  inherent  breakouts  of  second  and  sometimes  third 
level  outcomes.  This  provided  insight  and  assisted  the  professors  in  understanding  what 
underlying  or  inherent  knowledge  and  skills  were  required  for  a complex  cluster  of 
difficult  primary  learning  outcomes  - in  other  words,  the  knowledge  and  skills  inherent  to 
a single  complex  primary  outcome.  Therefore,  readers  may  be  amazed  at  the  number  of 
changes  that  resulted. 

Usually,  the  primary  statements  would  be  used  on  syllabi  or  other  reporting  documents, 
but  the  analysis  and  breakout  of  second  or  third  level  learning  statements  provided  a great 
learning  experience  for  professors  and  led  them  to  design  and  then  engage  students  in 
more  intentional,  thoughtful,  and  higher  quality  learning  experiences.  This  analysis  and 
process  can  lead  to  more  astute  teaching  and  student  learning,  student  assessments, 
instructional  choices,  learning  process  decisions,  and  more.  Remember,  each  course  is 
not  required  to  address  every  national  standard  or  outcome,  but  instead  the  standards  or 
outcomes  of  focus  selected  should  be  addressed  well.  It  is  important  that  they  understand 
individual  course  versus  program  requirements,  that  there  is  a cumulative  effect  across 
courses  for  the  entire  program,  that  the  overall  program  is  required  to  address  all  national 
standards  or  outcomes,  not  any  single  course;  therefore,  many  standards  will  be  addressed 
across  multiple  courses.  However,  particular  standards  may  be  addressed  in  only  one  or 
two  courses  across  the  program,  depending  on  content,  depth,  program  level,  (e.g., 
introductory  or  capstone  course).  Professors  sometimes  mistakenly  strive  to  address  all  or 
too  many  outcomes;  thus  the  course  content  can  become  weak  or  superficial.  Finally, 
when  identifying  the  objectives  or  outcomes  listed  below,  an  * is  used  where  one 
objective  or  outcome  covers  more  than  one  ABET  outcome  or  NAIT  standard  or  where 
there  is  a greater  total  of  “Is”  than  the  total  in  the  number  in  parentheses  (4).  The 
determining  factor  is  the  level  of  coverage  of  content. 

Regarding  outcomes,  it  is  important  to  note  that  the  professors  analyzed  the  engineering 
or  technology  course  content  for  embedded  NIU  General  Education  Goals.  This  analysis 
led  them  to  more  deeply  understand  why  students  fail  to  perform  well  in  their  courses  if 
they  do  not  come  to  the  course  with  the  appropriate  general  education  knowledge  and 
skills  that  are  the  underlying  foundation  for  the  engineering  and  technology  content.  The 
program  leader  revealed  the  strong  relationships  between  NIU  General  Education  Goals 
(outcomes)  and  the  ABET  and  NAIT  standards  or  outcomes  by  aligning  and  inserting 
them  into  a worksheet.  That  made  it  much  easier  and  more  efficient  for  the  professors  to 
see  the  direct  relationships,  then  to  consider  the  importance  of  acknowledging  the 
embedded  general  education  goals/outcomes  as  part  of  their  course  content,  and  to  realize 
that  even  though  they  are  teaching  engineering  or  technology  courses,  they  are  actually 
concurrently  continuing,  extending,  expanding,  and  deepening  the  learning  of  general 
education  content  in  the  context  of  engineering  and  technology.  This  was  extremely 
important.  Our  professors  intuitively  knew  this  but  had  never  “studied”  the  connections, 
mapped  the  connections,  or  included  the  general  educational  goals  aligned  beside  their 
engineering  or  technology  outcomes.  They  had  also  never  thought  of  themselves  as 
continuing  the  learning  of  the  general  education  knowledge  and  skills  in  engineering  and 
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technology  content.  They  considered  the  general  education  math,  science,  and 
communication  knowledge  and  skills  as  prerequisites  and  only  dealt  with  them  when 
students  did  not  have  the  knowledge  or  skills  needed  to  perform  on  the  engineering  and 
technology  content.  Now  the  professors  understand  that  they  actually  continue  the 
learning  of  the  mathematics,  science,  communication,  etc.  content  in  the  engineering  and 
technology  context.  The  chart  below  reflects  the  2005  course  in  black  and  the  changes  for 
the  2006  course  in  red.  The  professors  improved  the  outcomes  and  connections  and  are 
committed  to  greater  depth  of  change  for  the  future.  This  was  a very  successful  program 
component,  resulting  in  significant  learning  and  change. 

Assumptions 

Beware  of  assumptions  when  scanning  the  chart  below  and  noting  that  one  or  more 
course  outcome  numbers  did  not  seem  to  change.  For  example,  one  professor’s  number 
of  outcomes  did  not  change  from  2005-2006;  however,  the  quality  of  the  outcomes  for 
2006  was  very  significantly  different  and  improved.  Also,  that  professor’s  four  primary 
outcomes  were  broken  out  into  second  and  third  level  outcomes.  Again,  for  example,  one 
primary  outcome  inherently  encompassed  five  secondary  outcomes,  with  each  of  those 
broken  out  into  a third  level.  Thus  the  quality  in  content,  linkages,  and  assessments  was 
dramatically  different  and  greatly  improved  for  most  of  the  courses. 
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Table  BJ 

Ll.c.l:  Slant 

ards  ABET-Engineering  Outcomes 

(Fall  2005  and  Fall  2006  courses)  (5  engineering  professors) 

a.  apply 

b.  design/ 

C. 

d. 

e. 

f. 

g- 

h. 

i. 

j- 

k. 

math, 

conduct 

design  system, 

function  on 

identify,  formulate. 

understand 

ability  to 

understand  impact 

recognition  of 

knowledge  in 

ability  to  use 

science, 

experiments; 

component. 

inter- 

solve  engineering 

professional, 

communicate 

eng.  Solutions  on 

need  for,  and 

contemporary 

techniques. 

engineering 

analyze, 

process-given 

disciplinary 

problems 

ethical 

effectively 

global,  economic. 

ability  to 

issues 

skills,  and 

interpret  data 

constraints, 

teams 

responsibility 

environment,  society 

engage  in  life- 

modern 

etc. 

long  learning 

engineering 

tools 

Fall  2005  and  Fall  20 

06  Courses  - 

ABET  Outcomes 

5 + 

2+ 

4 + 

1+ 

4+ 

1-H 

1-H 

1-H 

2-h  1-h 

4-h 

4-h 

Ipartial 

1+partial 

1-1-  (written 

1 

(to  small 

1-H 

(no 

(could  do 

INR 

reports  only) 

INR 

effect) 

(students  don’t 

DOE) 

lots  more) 

INR 

2c 

use  unless 
asked  to) 

INR 
1 c 

1 c 

3c 

Ic 

2c 

3c 

Ic  (minor) 

2c 

Ic 

5+ 

2-1- 

4-1- 

2-1- 

5+ 

None 

3+ 

1-1- 

2-h 

4+ 

5-1- 

(6-11)  1-4 

1-1 

1-  2 

1-1 

*-8 

1-1 

1 - none 

* - none 

* . 2 

*-3 

(3/6-5)  1-5 

1-1 

1-5 

*-l 

*- 1 

1-1 

(4-4)  1 - 1 

* -none 

1 - none 

* - none 

1-1 

* - none 

*-l 

❖ 

❖ 

1 - 1 

(4-5)  1-f 

1-5 

3 

1-5 

* - none 

1-3 

* - none 

* - 1 

* - 1 

1-2 

(4-5)  1-2 

1-1 

1-1 

* - none 

1 - 1 

1 - 1 

Legend:  + = yes-okay;  c = need  to  consider;  other  notes 
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Table  B.ll.c.2:  Standards  ABET/TAC/NAIT-Engineering  Technology  & 
(2  engineering  technology/technology  professors) (Fall  2005 


Industrial  Technology 
and  Fall  2006  courses) 


mastery  of 
knowledge, 
techniques, 
skills, 

modern  tools 


b. 

ability  to 

apply 

current 

knowledge; 

adapt  to 

emerging 

applications 

of  math, 

science, 

technology 


c.  ability  to 
conduct, 
analyze, 
interpret 
experi- 
ments; 
apply 

experiment 
al  results  to 
improve 
processes 


d. 

ability  to 

apply 

creativity 

in  design 

of  systems, 

components, 

processes 


ability  to 
function 
effectively 
on  teams 


f. 

ability  to 

identify, 

analyze, 

solve 

technical 

problems 


g- 

h. 

i. 

j- 

k. 

1. 

m. 

n. 

o. 

ability  to 

ability 

recognize 

ability 

respect  for 

commit 

ability  to 

ability 

ability 

commu- 

to  com- 

need  for. 

to 

diversity; 

to  quality, 

program 

to  use 

to 

nicate 

municate 

ability  to 

under- 

know- 

timeliness. 

computers 

modern 

manage 

effectively 

effectively 

engage  in 

stand 

ledge  of 

continuous 

and/or  use 

labor- 

projects 

writing 

orally 

lifelong 

profes- 

contempor 

improve- 

computer 

atory 

effect- 

learning 

sional. 

ary 

ment 

application 

tec- 

tively 

ethical, 

profession- 

s 

niques, 

social 

al,  societal, 

effectively 

skills. 

responsi 

global 

equip- 

bilities 

issues 

ment 

effect- 

ively 

P- 

ability 

to 

design, 

mani- 

pulate, 

manage 

industri 

al 

systems 

q- 

ability 

to 

manage 
or  lead 
person- 
nel 

effect- 

tively 


Fall  2005  and  Fall  2006  Courses  - ABET/TAC/NAIT  Outcomes 


2+ 

2+ 

2+ 

2+ 

1+ 

Ic 

2+ 

1+ 

Ic 

2+ 

1+ 

Ic 

None 

2+ 

2+ 

1+ 

Ic 

2+ 

1+ 

Ic 

None 

1+ 

Ic 

2+ 

1+ 

Ic 

1+ 

9 

not 

sure 

1+ 

Ic 

1+ 

2+ 

1+ 

2c 

1+ 

1+ 

Ic 

1-1- 

1-1- 

Ic 

None 

P- 

1 no 
re- 
sponse 
Ic 

None 

q- 

Ino  re- 
sponse 
le 

None 

P- 

(5-6) 

* -NR 

1-6 

1-5 

1- 1 

* - 1 

1-4 

6 

* . 4 

2 

1 

2 

1 

1-1 

NR-NR 

q- 

(6-19) 

*-NR 

1-10 

1-5 

*-5 

* -3 

1-6 

1 

4 

1-one 

1 

*-none 

^"none 

NR-NR 

Legend:  + = yes-okay;  c = need  to  consider;  other  notes 
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Table  B.ll.c.3:  NIU  General  Education  Goals 


(Fall  2005  and  Fall  2006) 


(7  professors  across  engineering  and  technology) 


Writing 

Speaking 

Listening 

Quantitative 

Reasoning 

Use  of 

Resources- 

Technology 

Historical 
Development 
Of  Culture 

Significance 
of  Arts 

Cultural 

Traditions 

Philosophical 

Ideas 

Methods  in 
Science 
Methods  in 
Social  Science 

Interrelatedness 

Across 

Disciplines 

Social 

Responsihility 

Citizenship 

C C+ 

Earlier  it  was 
only  lab 
reports.  In 
fall  06,  they 
had  to  write 
reports  for 
three  PA 
tasks. 

C C+ 

Presentation 
of  PA  tasks 

+ + 
Listen  to 
guest  speaker, 
professor, 
fellow 
students 
during  PA 
task 

presentations 

+ C+ 

Homework, 

exams,  PA 

tasks  - all 

involved 

quantitative 

reasoning 

+ + 
In  addition 
to  lahs  that 
required 
using  many 
resources, 
had  to  use 
outside 
resources 
for  PA 
tasks. 

NR  NA 

NR  NA 

NR  NA 

NR  NA 

NRC 

May  consider 
more  interaction 
with  EE  for 
signal  processing 

NR  C+ 

Discussed  issues 
such  as  energy 
conservation, 
noise,  pollution, 
ethics,  etc. 

C+ 

Did  consider 
and  add,  still 
needs 

improvement; 
will  keep 
working  on  it 

C 

9 

C+ 

Strong,  but 
could  be  better 

+ 

I’m  quite 
pleased 

NA 

NA 

C- 

C+ 

C 

C 

Ok  + 

Project, 

exams, 

homework, 

using  MS 

Word 

C + 

Oral 

presentations 

with 

PowerPoint 

c c- 

Ok  + 

Material 
requires  this 

Ok  + 

Software 

and 

computer  to 

solve 

problems 

NR  NA 

NR  NA 

NR  NA 

C NA 

Ok,  C+ 

Examples, 
exercises  with 
topics  from 
other  disciplines 

C C- 

+ + 
PAs  and 
homework 

+ + 
Pas  and 
discussion 
sessions 

+ + 
Lectures,  case 
studies, 
discussion 
sessions 

+ + 
Problem 
solutions, 
homework, 
midterm  and 
final  exams 

+ + 
PAs  and 
homework 

C NA 

C NA 

C NA 

+ + 
Problem 
solutions, 
homework, 
midterm,  final 
exams,  and  PAs 

+ + 
Case  studies  and 
PAs 

+ C- 

C - + well 

addressed 
through  PA 
reports 

+ NA- 

possible  to 
include  for 
future  course 

C + 

lectures 

+ + 
addressed  in 
project  design 
decisions 

C + 

well 

addressed 

through 

project 

design 

decisions 

NR  NA 

C + 

addressed 
through 
project  design 

NR  - NA 

C + 

well  addressed 
through  project 
design 

C C+ 

to  some  extend 
when  making 
design  decisions 

c c- 

possihle  to 
include  for 
future  courses 

+ + 

Performance 

Tasks 

+ + 
Group 
learning  and 
interactions 

+ + 

Group 

interactions 

+ + 
Performance 
tasks,  lah 
demonstrations 

C NA 

+ C+ 

C + 

Lectures 

+ C+ 

+ NA? 

C C+ 

+ C+ 

C + 

C C- 

C C- 

+ N+ 

+ + 

+ NR 

C NA 

C NA 

+ NA 

C NA 

C N\A 

Legend:  + addressed  well;  NA-does  not  really  apply  in  professor’s  opinion;  C-  do  not  do  it,  but  still  need  to  consider  adding  it  in  as  professor  continues  to 
make  changes;  C+  did  consider  and  add  in;  still  needs  improvement  and  professor  will  keep  working  to  improve  or  add; 


42 


Research  Semester  Results  on  Teaching  Styles  (Fall  2005  and  Fall  2006) 

During  the  initial  course  analysis,  professors  analyzed  their  2005  courses  for  use  of  teaching  styles.  They  referenced  Mosston  and  Ashworth 
(1990)  only.  At  the  end  of  the  research  semester,  professors  were  provided  the  same  Teaching  Styles  list  by  Mosston  and  Ashworth  and  also 
Grasha’s  (1996).  They  were  asked  to  consider  which  styles  were  used  during  the  research  semester.  The  responses  ranged  from  check-offs 
to  comments.  Mosston  and  Ashworth’s  styles  are  compared  for  the  2005  and  2006  courses  on  (Table  5)  below  the  one  for  Grasha  (Table  4). 
The  results  from  considering  Grasha’s  are  presented  in  the  chart  immediately  below  in  narrative.  Some  professors  estimated  how  many 
times  they  used  each  style;  others  made  comments  about  the  ones  they  choose;  and  others  did  both.  All  professors  made  comparisons  using 
Mosston  and  Ashworth’s  teaching  styles,  but  some  professors  also  considered  Grasha’s  Mosston  and  Ashworth  were  provided  during  the 
initial  analysis  early  in  the  program.  Later  in  the  program  we  were  trying  to  present  them  with  varying  options  and  perspectives.  Therefore, 
they  were  also  exposed  to  Grasha’s  styles.  The  most  important  aspect  of  this  reporting  activity  was  to  reinforce  consideration  of  teaching 
styles  and  to  stimulate  a broader  repertoire  of  teaching  styles  or  the  use  of  a greater  variety  of  teaching  styles  in  their  courses.  Grasha  is 
presented  first.  Note:  Outcomes  vary  across  professors,  so  the  two  tables,  Grasha  and  Mosston  and  Ashworth,  reflect  which  teaching 
styles  are  used  across  the  total  of  individual  primary  course  outcomes  by  professor.  Outcomes  are  presented  by  number  only  in  left 
column.  This  program  component  was  very  successful  in  that  professors  varied  their  teaching  styles  beyond  those  used  in  their  2005 
courses. 


Table  B.ll.c.4:  Student  Learning  Outcomes  & Teaching  Styles  (Fall  2006)  (7  professors  across  engineering  and  technology) 


# of  Outcomes 

Expert 

Formal  Authority 

Personal  Model 

Facilitator 

Delegator 

6 

Yes  -4 

Yes -5 

NR 

Yes -6 

Yes -3 

3-6 

Used  Felder  formally 

NR 

NR 

NR 

NR 

Responded  to  Kolb 

4 

Yes 

Yes 

NR 

NR 

NR 

Used  Kolb  formally 

4 

Yes,  but  used  less  this  time 

Used  for 
fundamentals 

NR 

Used  with  PA  tasks, 
especially  1 & 2 

Used  for  final  PA 
task  #3 

Used  Kolb  formally 

5 

No  Response  to  Grasha 

NR 

NR 

NR 

NR 

Responded  to  Kolb 

6 

No  Response  to  Grasha 

NR 

NR 

NR 

NR 

Responded  to  Kolb 

4 

Yes 

Yes 

No 

Yes 

Yes 

Responded  to  Kolb 
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Table  B.ll.c.5:  Student  Learning  Outcomes  & Teaching  Styles  (Fall  2005  and  Fall  2006) 

(7  professors  across  engineering  and  technology) 


#of 

Outcomes 

Command 

Practice 

Reciprocal 

Self- 

Check 

Inclusion 

Guided 

Discovery 

Convergent 

Discovery 

Divergent 

Production 

Learner 

Designed 

Learner 

Initiated 

Self 

Teaching 

6 

6+ 

yes -10 

6+ 

yes-4 

2c,  4+ 
yes  -5 

6c-lmin. 

6c 

5-I-,  Ic 
yes -20 

6c 

3c  (min) 
3c 

yes  -6 

6c 

6c 

yes  -1 

6c 

3-6 

6+ 

Less  than 
before 

6c 

Much 

more 

5c,  (l  little) 
a few 
times;  did 
not  guide 
oh. 

6c 

more 

than 

normal 

6c 

3-I-,  3c 
more 
than 
before 

lean  do  more 

1-I-,  4c 
more  than 
before 

5c,  llittle 

6c 

6c 

more 

than 

before 

6c 

more 

than 

before 

4 

2ok 

yes 

2ok,c 

yes 

2c 

yes 

2c 

2c 

NR 

2c 

yes 

NR 

NR 

NR 

NR 

4 

4+ 

used  less 
this  time 

4+ 

2c,  2+ 
used  much 
more 
during 
oral 

discussions 

4c 

4c 

new,  somewhat 
accomplished 
through 
implementation 
of  the  rubrics 

4c 

4c 

used  much 
more  - a 
lot  through 
PA  tasks, 
discussions 

4c 

4c 

4c 

4c 

5 

5+ 

5some 
used  appx. 
6 times 
mainly/PAs 

NR 

4-l-(lsome) 

4c,  INR 

3c,  2NR 

3c,  2NR 

NR 

used  appx. 
12  times  / 
problem 
solving 

NR 

4NR,lc 

NR 

6 

5+,  Ic 
yes  - 20 

5+,  Ic 
yes  - 6 

6c 

yes  - 6 

6c 

6c 

3-h,  2c 

5c,  1-1- 
yes-3 

5-h,  Ic 

5c,  1-1- 

6c 

yes -3 

5-I-,  Ic 

4 

4+ 

no;  but,  yes 
with  other 
styles  below 

4+ 

yes,  when 
solving 
problems  in 
class 

4+,  c 

within  the 
group;  but 
without 
professor 
supervision 

4c 

yes; 

feedback 

within 

group 

NR 

yes 

4c-h 

4c+ 

yes 

4c 

yes, 

sometimes; 
when  the  was 
design 
problems 

4c 

yes 

4c 

yes,  but 
with  some 
guidelines; 
instruction 
is  given 

4c 

yes,  with 
projects; 
but  not  with 
deep 

consultation 
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Research  Semester  Teaching  Models  (Fall  2006) 

In  the  table  below,  the  professors’  analysis  of  their  Fall  2005  eourse  is  eompared  to  what  they 
indieated  aetually  oeeurred  in  the  same,  but  signifieantly  revised,  eourse  during  the  researeh 
semester  of  Fall  2006.  The  Fall  2005  course  is  presented  first  in  black,  and  beneath  that  information, 
the  teaching  models  used  during  the  Fall  2006  course  during  the  research  semester  are  presented  in 
red.  There  are  24  teaching  models;  therefore,  the  complete  list  is  presented  in  two  charts;  models  are 
identified  across  the  first  row.  The  numbers  in  black  represent  what  they  would  consider  using, 
acknowledging  that  in  the  2005  course  those  were  not  in  use.  The  number  or  comments  in  red 
represent  what  they  felt  they  tried  in  the  2006  experimental  course. 

Although  it  may  appear  that  professors  selected  only  a few  new  models  to  use  during  their 
experimental  course  in  2006,  remember  there  are  24  different  models  to  consider.  They  were 
encouraged  to  select  just  a few  models  to  try  out  in  the  2006  courses,  and  then  to  add  other  models 
gradually  in  consecutive  semesters.  Thus,  each  professor  chose  a few  models  to  try  that  were 
different  than  the  most-often  used  “lecture”  model. 

This  aspect  of  the  program  was  successful:  professors  were  exposed  to  24  teaching  models.  They 
used  this  initial  approach  to  analyze  what  models  they  felt  were  used  in  the  2005  course.  Most  of 
them  had  no  previous  knowledge  of  these  models  nor  had  they  considered  “teaching  models”  at  all, 
even  those  who  had  attended  teaching  workshops.  Several  had  been  exposed  to  “cooperative 
learning,”  one  of  the  teaching  models  below,  but  had  not  used  the  model  formally  at  all  and  only 
weakly  structured  informally. 

During  the  teaching  model  program  component,  the  professors  studied  the  24  models  more  in  depth; 
this  was  after  the  initial  analysis  with  a list  and  brief  descriptions.  The  worksheet  used  as  a study 
guide  along  with  the  Joyce,  Weil,  and  Calhoun’s  (2004)  Models  of  Teaching  book  reveal  how  they 
felt  about  each  model  and  whether  they  felt  each  model  had  potential  for  use  in  teaching  their 
course.  That  worksheet  is  presented  later,  as  it  is  a formal  segment  of  the  program  for  the 
redevelopment  of  the  course.  However,  when  reviewing  the  worksheet,  each  professor’s  comments, 
and  then  the  comments  after  the  experimental  course,  one  can  see  the  growth,  comments,  or 
questions. 

After  the  research  semester.  Fall  2006,  we  returned  their  initial  analysis  and  the  study  worksheet 
and  asked  them  to  note  which  teaching  models  they  felt  they  had  actually  used  during  the 
experimental  semester.  Did  they  use  the  ones  that  they  expected  to  try  out?  Did  they  use  others  not 
expected?  The  red  numbers  below  labeled  2006  are  those  responses.  The  data  reveal  significant 
change,  considering  the  context  was  one  course  and  their  first  effort  to  expand  their  teaching  model 
repertoires. 

(See  Teaching  Models  In  Portfolio  Section  B.ll) 
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Table  B.ll.c.6:  Student  Learning  Outeomes  & Teaehing  Models 


(Fall  2005  & Fall  2006)  (7  professors  aeross  engineering  and  technology) 


# Out- 
comes 

Memory 

Progressive 

Part 

Advanced 

Organizers 

Lectnre 

Reciprocal 

Teaching 

Mastery 

Learning 

Cooperative 

Learning 

Graphic 

Organizers 

Concept 

Attainment 

Concept 

Formation 

Concept 

Presen- 

tation 

Conceptnal 

6 

6c 

6c 

5c,2-h 
yes  20-t- 

6-1- 

yes  10 

6c  (Imin.) 

5c,  1-1- 

3-I-,  3c 
yes  3 

3+,  3c  (Imin) 
yes  4 

6c  (Imin) 

6c  (Imin) 

6c 

3-1- ,3c 
yes  6 

3-6 

6c 

6c 

couple  times 
liked  it 

6c 

have  always 
used  it 

6-1- 

do  mnch 

less 

6c 

several 

times 

6c 

nsed,  not 

completely 

rigorons 

6c 

used  much 
more  & 
more  formal 

6c  (1  min) 
nsed  abont  as 
much  as  before 

5c,l-i- 

used...probabl 
y slightly  more 

6c 

6c 

nsed  a 
little 

6c 

used 

4 

NR 

2c 

yes  14 

2c 

yes 

2+ 

yes  14 

2c 

yes  7 

2c 

yes  10 

2ok,c 

yes  7 

2c 

NR 

NR 

2c 

yes 

2c 

4 

NR 

NR 

NR 

yes,  good 
response 

4-1- 

yes,  several 
times 

2c,  2+ 

NR 

2c,  2+ 

yes,  good 
response, 
more 

assessments 
taken  the 
time 

4c 

4c 

4c 

4c 

*Used 
with  *C, 
more 
often 
than 
before; 
now  I 
know 
what  this 
is  called. 

4-1- 

****  nsed 
with  * CP 

5 

2-h 

4-1- 

4-1- 

4+ 

20-used 
frequently 
to  deliver 
course 
materials 

4c 

3-used 
while 
execnting 
PAs; 
demon- 
strated good 
outcomes. 

NR 

4c 

3-  nsed  to 
enhance 
implementat 
ion  of  PAs 

4+ 

NR 

NR 

3c,inr 

4c 

6 

5c 

4-i-,2c 

15 

lectures 

4+,  2c 
3 theory 
linked 
with  lah 
demon- 
stration 

5-I-,  Ic 
20 

lectures 

6c 

6c 

6c 

6-group 

learning 

&PA#3 

5-i-,lc 

10-during 
lectures- 
visual  aids 

6c 

6c 

2c,  4-1- 
6- 

lectures 

on 

funda- 

ment 

4-f,2c 

4 

4c 

used,  but 
not  much 

4-1- 

yes 

4-1- 

yes 

4-1- 

some  parts 
lecture,  but 
not  majority 

4c 

did  this  with 
projects 

4c 

4-1- 

yes,  done 
with  PA 
projects 

4c 

yes,  every 
group  did  that 

4c 

4c 

4-1- 

4-1- 
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# Out- 
comes 

Inductive 

Deductive 

Inquiry 

Simulation 

Jurispruden 

tial 

Direct 

Instruction 

Training 

Synectics 

Psychomotor 

Metaphore 

Non- 

Direct 

Role 

6 

6c 

yes  4 

6c  (Imin) 

6C(2min) 

yes  5 

6C(2min) 

yes  2 

6c 

4+,2c(lmin) 

yes  8 

4+,  2c 
yes  5 

6c 

5NR,lc 
yes  2 

NR 
yes  1 

NR 
yes  2 

NR 
yes  2 

3-6 

6c 

used  a lot 

6c 

used  less 
than  before 

2+,  4c 
used  a 
lot 

6c 

extensive 

use 

6c 

6c 

1-I-,  5c 
about  as 
before 

6c 

6c 

some 

6c 

6c 

6c 

4 

NR^ 

NR 

2ok 

NR 

NR 

NR 

NR 

NR 

NR 

NR 

NR 

NR 

NR 

4 

4-t- 

used  much 
more; 
students 
responded 
well!  Used 
past  also. 

4-1- 

used  much 
more; 
students 
responded 
well!  Used 
past  as  well. 

NR 

4c 

NR 

NR 

NR 

done  some 
before;  use- 
ful for  pro- 
blem solving 
procedures; 
excellent  / 
eonceptuali- 
zation;  able 
to  discuss 
different 
approaches 
after  one  as 
presented. 

NR 

NR 

NR 

NR 

NR 

5 

NR 

NR 

4-1- 

7 

4c 

NR 

4c 

10  suitable 
for  certain 
topics 

Ic,  4NR 

NR 

NR 

NR 

3 

4c 

Ic,  3NR 

6 

6c 

9 - Pas  & 
assignments 

6c 

6c 

4+,  2c 

2 lab  demos 

6c 

2-1-,  4c 

15  lectures 
on  basics 

5c,  1-1- 

2 lab  demos 

6c 

5c,  l-H 

6c 

6-1- 

5c,  1-1- 

4 

4c 

yes,  hidden 
in  lecture 

4-1- 

yes,  but 
professor 
dos  that 
when 
needed 

4c 

4c 

yes,  students 
simulate 
perfor- 
mance of 
rubrics 

4c 

4c 

4c 

4c 

4c 

4c 

4c 

4c 

(Scarborough,  2006  based  on  Joyce, 


Weil,  & Calhoun,  2004) 
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Student  Learning  Outcomes  & Kolb  (1984)  Learning  Styles 

The  Chart  below  identifies  what  learning  styles,  according  to  Kolb’s  (1984)  styles,  the  professors  felt  they  were  providing  opportunity  for  students  to  use  in  both  the 
2005  and  2006  courses.  It  appears  that  more  attention  was  paid  to  learning  styles  across  professors  in  the  2006  experimental  course.  Two  professors  used  the  Kolb 
Learning  Styles  inventory  formally  with  the  entire  class,  and  a third  professor  used  the  Felder  Learning  Styles  Inventory  formally  with  his/her  class.  This  segment  of  the 
program  was  also  considered  very  successful,  as  it  greatly  enhanced  the  professors’  understanding  of  the  overall  focus  of  teaching  and  the  relationship  between  teaching 
styles,  teaching  models,  and  student  learning  styles.  Their  awareness  was  greatly  increased;  their  understanding  increased;  and,  their  commitment  to  working  on 
increasing  the  diversity  of  teaching  models  and  styles  to  better  engage  a broader  range  of  student  learning  styles  and  to  also  culminate  in  expanding  individual  student 
learning  styles  was  significant.  Below  are  reflections  from  the  three  professors  who  formally  used  LS  Inventories.  (See  Felder  notes) 


Table  B.ll.c.7:  Student  Learning  Outcomes  & Kolb  Learning  Styles  2005  & 2006  (7  professors  across  engineering  & technology) 


# Outcomes 

Concrete  Experience 

Abstract  Conceptualization 

Active  Experimentation 

Reflective  Observation 

6 

2c-minimal,  2c,  2+ 

5+,  Ic 

3c,  3+ 

6c 

Also  used  Felder’s 

SL,  IL,  VL,  VL,  AL,  RL, 

GL. 

yes 

yes 

yes 

yes 

3-6 

6+ 

3-h,  2c 

4+,  2c 

3-h,  1 not  so  much,  2c,  I a 

Used  Felder’s  only. 

Global  learners  did  better 

Intuitive  learners  did 

Visual  and  verbal  learners  did 

little 

The  notes  are  ture  for  the 

than  sequential  learners. 

Better  than  sensing  learners. 

equally  well. 

Reflective  learners  did 

concept  tests,  hut  less  so 
on  the  problem  solving 
tests.  See  write  un  below. 

better  than  active  learners. 

4 

NR 

2ok,c 

2ok,  c 

2c 

Formally  used  Kolb  with 
students  as  a way  to  show 
students  their  learning 
styles.  Will  use  it  next  time 
to  also  create  activities 
tailored  to  students’ 
distribution  of  L.  styles. 

yes 

yes 

yes 

yes 

4 

Formally  used  Kolb’s 

2c,  2+ 

4+ 

4+ 

4+ 

Inventory. 

Yes 

Yes 

yes 

yes 

Concentrated  effort  was 

made  to  have  activities 

to  engage  all  four  (4) 

learning  styles. 

5 

5+ 

INR,  I+,  3c 

INR,  Ic,  3-h 
yes 

4+,  Ic 

6 

No  response 

6c 

5+,  Ic 

5c,  1+ 

4+,  2c 

4 Also  used  Felder’s  SL, 

4+ 

4+ 

2c,  2+ 

2c,  2+ 

VL,  VL,  AL,  RL,  SL,  GL 

yes 

yes 

Legend:  Black-Kolb  2005  course  analysis;  Red-2006  course  analysis;  Blue-2006  course  using  Felder 
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Reflections  on  using  Felder  & Solomon  Learning  Styles 
B.D.  Collar 

In  the  fall  of 2006,  we  conducted  a research  project  experimentally  investigating  student  learning  in  an  introductory  engineering  mechanics  course.  As 
part  of  the  project,  we  administered  Felder  and  Soloman’s  Index  of  Learning  Styles.  The  survey  is  designed  for  engineering  undergraduates.  It  consists  of 
44  questions  aimed  at  illuminating  students’  preferred  modes  of  learning.  Felder  and  Soloman  characterize  student  learning  styles  with  four  dimensions: 

1.  active  V5.  reflective, 

2.  sensing  V5.  intuitive, 

3.  visual  V5.  verbal, 

4.  sequential  V5.  global. 

In  the  research  project,  we  randomly  split  the  class  into  two  groups.  With  one  group,  we  used  hands-on  manipulatives  to  present  many  of  the  concepts. 
The  second  group  is  a control  group  in  which  we  used  more  traditional  graphical  techniques  to  introduce  and  solidify  concepts. 

A5  it  turned  out,  there  was  no  statistically  significant  difference  in  the  two  groups’  performances  on  objective  performance  tests.  However,  when  we 
examined  the  data  more  closely,  we  did  find  an  interesting  distinction.  Electrical  engineering  students  in  the  experimental  group  did  significantly 
better  than  their  counterparts  in  the  control  group.  It  was  an  effect  not  present  in  the  mechanical  engineering  students,  who  make  up  the  bulk  of  the  class. 
In  fact  mechanical  engineering  students  in  the  control  group  tended  to  do  slightly  better  than  their  counterparts  in  the  experimental  group,  but 
not  by  a statistically  significant  margin. 

It  is  apparent  from  our  data  that  electrical  engineering  students  think  and  learn  differently  than  mechanical  engineering  students.  An  obvious  question  is 
what  makes  the  electrical  engineering  students  more  receptive  to  the  hands-on  teaching  strategy?  When  we  correlated  students’  learning  styles  to  exam 
performance,  we  found  that 

1.  Reflective  learners  tended  to  perform  better  than  active  learners. 

2.  Intuitive  learners  tended  to  perform  better  than  sensing  learners. 

3.  There  was  no  correlation  between  the  visual/verbal  dimension  of  learning  and  exam  performance. 

4.  Global  learners  tended  to  perform  better  than  sequential  learners. 

In  results  I,  2,  and  4 above,  the  p-values  were  all  less  than  0.002.  However,  all  correlation  coefficients  had  magnitudes  on  the  order  of  0.4.  Therefore, 
while  certain  learning  styles  showed  a tendency  for  better  performance,  it  is  clear  that  there  was  no  definite  one-to-one  correspondence.  So  are  the  more 
advantageous  learning  styles  more  prevalent  in  electrical  engineering  students?  The  answer  is  no. 

We  found  no  statistical  difference  between  the  learning  styles  of  electrical  and  mechanical  engineering  students.  In  the  study,  we  tested  for  several  other 
differences  between  mechanical  and  electrical  engineering  students  that  also  correlated  with  exam  scores.  We  were  not  able  to  find  any.  For  now,  the 
difference  is  a mystery. 
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Application  of  Kolb’s  Learning  Styles  to  ISYE370 
By  Reinaldo  Moraga 

I started  my  Operations  Research  class  —ISYE370—  by  giving  the  Kolb’s  learning  inventory  test  to  my  students  in  such  a way  that  they  and  I became  aware  of  the  type  of 
learning  style  they  used  to  learn  and  which  other  styles  they  were  able  to  pursue.  In  addition,  Kolb’s  learning  styles  helped  me  to  improve  the  delivery  of  the  teaching. 

The  test  was  given  to  each  student  in  the  first  class  after  the  presentation  of  the  syllabus.  Then  I explained  to  them  the  importance  of  recognizing  their  preferred  learning 
style  and  how  this  information  could  be  used  for  them  and  me  to  enrich  the  learning  environment  in  the  classroom.  In  addition,  I tried  to  connect  the  importance  of  this 
tool  with  their  professional  career  in  terms  of  communicating  in  the  workforce  and  collaborating  in  groups.  Step  by  step,  I went  through  the  booklet  to  let  them  know  how 

to  fill  the  questionnaire  and  interpret  the  results.  The  students  were  inclined  to  think  that  there  was  a correct  outcome  for  this  test.  Therefore,  I had  to  make  clear  that 
this  was  only  a way  to  diagnose  a r>referred  style  of  learning.  Finally,  I asked  to  take  the  test  home,  answer  the  questionnaire,  and  next  class  give  me  a brief  essay 
reporting  (a)  their  preferred  learning  style,  (b)  actions  they  could  take  to  expand  their  learning  into  other  styles,  and  (c)  which  type  of  activities  in  this  class  could 
produce  connection  with  their  preferred  and  other  learning  styles. 

Most  of  the  students  were  able  to  identify  their  preferred  learning  style.  To  expand  their  learning  styles,  most  of  them  reported  activities  such  as  “exploring  the  world 
around,”  “reading  more  books,"  “doing  more  [hand  work],’’  “being  more  sensitive  to  people’s  feelings,”  “trying  to  make  the  subject  fun  while  learning,”  etc.  Some 
interesting  comments  on  how  to  connect  their  learning  styles  with  my  class  were  “by  becoming  personally  involved  and  influencing  the  others  to  work  together,  ” “to 
have  a review  session  or  a guide  study,  ” and  “to  gather  into  groups  to  think  out  problems.  ” 

I found  this  activity  relevant  because  we  may  use  Kolb’s  test  to  help  us  identify  our  strengths  and  weaknesses  as  instructors,  recognize  our  students'  preferred  styles,  use 
teaching  techniques  to  require  all  learning  styles,  and  encourage  our  students  to  extend  into  other  styles.  Of  particularly  interest  to  me  as  instructor  was  to  learn  the  use 
of  the  learning  cycle  to  design  some  of  my  “lectures.  ” The  learning  cycle  consisted  of  four  questions:  why?,  what?,  how?  and  what  if?  (Harb,  Durrant,  & Terry,  1993.)  I 
tried  to  emphasize  in  my  lectures  the  answers  to  these  questions  because  in  that  way  I could  reach  most  of  the  different  learning  styles  of  my  students.  This  framework 
opened  my  eyes  to  the  importance  of  Kolb’s  learning  styles,  and  because  of  its  practical  applications  in  teaching,  I would  like  to  keep  using  Kolb  learning  cycles  as  part 
of  my  other  classes  I teach  for  the  College  of  Engineering. 

Reference: 

Harb,  Durrant,  & Terry,  (1993).  Use  of  the  Kolb  learning  cycle  and  the  4MAT  system  in  engineering.  Education,  Journal  of  Engineering  Education,  70-77. 
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Use  of  Kolb’s  Inventory  of  Learning  Styles 
CITL  - lENG  475  Fall  2006 
Regina  Rahn 

The  Kolb  Learning  Style  Inventory  was  administered  to  students  in  the  Fall  2006  lENG  475  Decision  Analysis  class.  They  completed  the 
questionnaire  and  interpreted  their  learning  styles.  We  discussed,  as  a class,  the  strengths  of  each  learning  style  and  talked  about  the  types 
of  activities  that  were  useful  for  facilitating  learning  of  each  type.  The  idea  was  to  set  a premise  for  the  assessment  and  instructional 
activities  that  would  be  implemented  during  the  semester. 

In  addition,  we  discussed  ways  that  individuals  could  use  the  knowledge  of  their  learning  styles  to  expand  the  ways  in  which  they  learn  to 
incorporate  other  styles.  The  discussion  included  the  use  of  group  work  (cooperative  learning)  and  peer  review  as  ways  to  aid  in 
accomplishing  this  goal. 

A graduate  student  used  this  as  one  of  the  bases  for  her  graduate  project.  The  project  was  completed  at  the  end  of  the  semester.  The  lENG 
475  students  were  surveyed  about  their  thoughts  in  regard  to  the  use  of  the  learning  style  inventory.  The  responses  were  extremely  positive, 
and  they  definitely  saw  the  value  in  the  exercise. 

The  Decision  Analysis  class  also  posed  a unique  opportunity  for  discussions  surrounding  learning  styles.  We  investigated  relationships 
between  learning  styles  and  peoples  ’ attitude  toward  risk,  which  is  a key  element  in  the  course  subject  matter.  I intend  to  continue  utilizing 
learning  style  inventories  in  my  courses. 


Student  Learning  Outcomes  & Bloom  & Dale  (Fall  2005  and  Fall  2006) 

The  professors  analyzed  their  student  learning  outcomes  against  Bloom’s  Learning  Dimensions  and  Dale’s  Cone  of  learning.  The  analysis  of  the  2005  course  is 
presented  in  black  below  and  also  as  a composite,  number  of  outcomes  achieving  what  level  on  Bloom’s  and  Dale’s  models.  The  2006  course  analysis,  however, 
is  presented  in  red.  Dale’s  levels  are  presented  by  number  of  outcomes  and  level  of  the  Cone.  For  Bloom,  each  outcome  is  listed  at  the  level  achieved.  This 
program  component  was  also  successful.  The  professors  really  seemed  to  grasp  Bloom’s  intentions,  whether  traditional  or  revised.  They  not  only  benefited  from 
using  it  as  an  analysis  tool,  but  in  the  later  re-development  of  their  courses.  They  also  grasped  Dale’s  intentions  about  passive  versus  active  learning.  These 
models  seemed  to  build  good  initial  awareness,  which  deepened  as  they  used  them  as  tools  more  and  more,  beginning  with  the  initial  2005  analysis  and  then  as  a 
metric  for  the  re-development  of  the  2006  courses.  There  was  significant  change  in  the  quality  of  their  student  learning  outcomes.  The  professors’  student 
learning  outcomes  were  developed  and  written  to  achieve  higher  cognitive  processing  levels  on  Bloom’s  Cognitive  Dimension.  T he  outcomes  also  reflective 
higher  quality  in  that  they  reflected  more  active  learning.  The  outcomes  reflected  a potentially  higher  level  of  critical  thinking  as  well.  This  program  component 
resulted  in  significant  change  and  left  them  with  simple  tools  to  use  as  metrics  for  ongoing  change  and  quality  checks. 
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Table  B.l  l.c.8:  Student  Learning  Outcomes  & Bloom  & Dale  (Fall  2005  and  Fall  2006) 
(7  professors  across  engineering  and  technology) 


# Outcomes: 

Dale's  Cone 

Knowledge 

Comprehension 

Application 

Analysis 

Synthesis 

Evaluate 

Critical 

05  reported  as  composite 

Levels  : P 

Remember 

Understand 

Apply 

Analyze 

Evaluate 

Create 

Thinking  Level: 

06  reported-  specific  outcome 

AA+ 

LM  H 

(1)  6 outcomes 

NR 

1+,  5NR 

2-I-,  4NR 

2+,  4NR 

Ic,  5NR 

1+,  5NR 

6NR 

2Lm  3M,  IH 

composite 

1-11 

1,  3,  7.  8.  9 

2,  5,  6,  7. 10, 11 

1,4 

1-11  numbers  1-11 

(8-10)  A-i- 

3P,  lA 

6-1- 

6-1- 

3-I-,  3c 

5c,  INR 

6c 

6c 

2L,  2L+,  IL/M,  IM; 

(2)  3/6 

2A- 

1-3/6 

2Mc 

1-5 

5 

(10)  At- 

2PA-C 

2-i-c 

2-i-c 

2-I-C 

2-I-C 

2-I-C 

2c 

2Mc 

(3)  2 

1-2 

1,2 

2 

(8-10)  At- 

2A,  2A-I- 

4-1- 

4-1- 

4-1- 

4-1- 

Ic,  l-i--,  2-1- 

4c 

2M,  2H 

(4)  4 

NR 

NR 

NR 

NR 

NR 

NR 

NR 

2P,  2A,  1A+ 

2+,  3NR 

2NR,  3+ 

2NR,  3+ 

NR 

NR 

NR 

3M 

(5)  5 

1(6); 

19 

2-3  (9) 
4(6) 

5-19 

(9- 10)  At- 

NR 

NR 

NR 

NR 

NR 

NR 

4P,2A 

3(2) 

2-I-,  4c 

3-I-,  3c 

5-I-,  Ic 

4-I-,  2c 

5-I-,  Ic 

4c,  2-1- 

3L,  2M,  IH 

(6)  6 

4 (1,3,5) 

1,  2,  3,  5 

4 

6 

6 

8-9  (1) 
8(1) 

IP,  IP-A, 

4-1- 

4-1- 

1-i-c,  3-1- 

Ic,  let-,  2-1- 

2c,  2-1- 

2c,  2-1- 

2c,  2c-i- 

2A-P 

1 

2 

3,4 

(7)  4 

4 

1-4  (1-10) 

Bloom  (1956);  Anderson  & Krathwohl  (2001);  Legend  for  Blooms  levels:  NR  = no  response;  number  + = number  of  outcomes  at  that  level;  +c  = okay,  but  still  need  to  consider;  c = 
need  to  consider  achieving;  c+  = some  positive  accomplishment,  hut  still  needs  work  (e.g.,  outcome  number  reported  by  each  Boom  level) 

Dale  (1969):  Legend  for  Dale’s  levels:  9-10  = active  learning-doing  level;  8 = active  learning-participating;  3-7  = visual  receiving/passive;  2-1  = verbal  receiving-passive, 

(e.g.,  outcome  number  - level) 
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CLASSROOM  RESEARCH  SEMESTER 
RESULTS  ON  THE  SCHOLARSHIP  OF  TEACHING 
Jerry  Gilmer,  Ph.D.  and  Jule  Dee  Scarborough,  Ph.D. 

. (See  B.13  for  manuscript  drafts  presenting  their  individual  research  results.) 

Each  faculty  member  participated  in  the  culminating  research  semester,  where  they  performed 
experimental  research  on  teaching  and  learning.  Before  that  final  aspect  of  The  Faculty 
Development  Program  on  Teaching  and  Learning,  the  professors  engaged  together  to 

• analyze  their  courses  for  quality  of  disciplinary  knowledge  content, 

• analyze  the  embedded  general  education  knowledge  within  the  engineering  and 
technology  content 

• analyze  the  variety  of  teaching  models  and  styles  used  throughout  their  courses 

• analyze  the  range  of  student  learning  styles  made  possible  in  the  course 

• analyze  the  level  of  Bloom’s  Cognitive  Process  Dimension  and  Dale’s  active  vs. 
passive  learning  achieved  by  the  course  and  students 

• analyze  the  quality  of  student  assessments,  e.g.,  tests 

• analyze  the  strength  of  the  connections  between  the  national  student  learning 
outcomes  and  the  student  learning  outcomes  for  the  courses 

• analyze  the  strength  of  the  connections  between  the  course  content  and  student 
assessment(s)  and  items 

• redevelop  their  2005  courses  and  develop  new  student  learning  outcomes  for  the  2006 
courses 

• develop  new  tests  and  performance  assessments  with  rubrics 

• develop  grading  structures  and  criteria 

• select  new  teaching  models  and  styles  to  use  in  the  redeveloped  2006  course 

• make  choices  about  instructional  strategies  that  broadened  the  range  of  possible 
student  learning  styles 

• learn  about  educational  or  social  science  experimental,  classroom  research  on 
teaching  and  learning 

• interact  with  program  leaders  to  design  an  experimental  study  on  teaching  and 
learning  related  to  what  they  had  learned  throughout  the  faculty  development 
program 

• perform  experimental  classroom  research  with  their  students 

• implement  new  teaching  strategies,  models,  processes,  techniques,  and  more 

• prepare  research  manuscripts  to  submit  to  professional  journals  detailing  the 
results  of  their  research 

Therefore,  as  one  program  component  in  the  CELT  Faculty  Development  Program  on  Teaching 
and  Learning,  each  of  the  seven  professors  carried  out  a formal  research  project  in  their 
classrooms  during  the  Fall  2006  semester,  with  each  research  project  conforming  to  the 
professors’  specific  areas  of  interest  stemming  from  the  program  components  in  the  faculty 
development  program.  We  consider  this  a critical  aspect  of  our  program.  Professors  began  the 
program  by  building  a knowledge  foundation  about  teaching  and  learning;  used  that  knowledge 
to  analyze  their  courses  and  develop  new  ones,  including  educational  products  to  use  with 
student;  and  then  participated  in  the  culminating  experience,  where  they  taught  the  newly 
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developed  courses  using  new  educational  strategies,  methods,  and  procedures  as  well  as  new 
products  and  engaged  in  experimental  research  fir  the  first  time.  Some  of  the  projects  included 
comparing  cooperative  performance-based  procedures  to  individual  performance-based 
procedures,  comparing  hands-on  manipulative  procedures  to  graphical  procedures  in  problem- 
based  learning  projects,  examining  the  effects  of  the  order  of  administration  of  performance 
assessment  and  a traditional  assessment,  and  others.  See  the  full  range  of  research  questions 
below  followed  by  the  results.  All  experimental  designs  are  considered  true  experimental 
designs,  with  each  involving  at  least  two  experimental  groups,  random  assignment  of  students  to 
each  group,  and  both  a pretest  and  a posttest.  Of  the  seven  studies,  only  one  exhibited 
statistically  significant  results  (cooperative  performance-based  procedures  appeared  to  result  in 
more  learning  than  individual  performance-based  procedures). 

From  the  perspective  of  each  research  study,  the  absence  of  statistical  significance  may  be 
disappointing  to  the  professors.  However,  from  the  perspective  of  the  Initiative,  the  actual 
completion  of  these  research  studies  indicates  a high  level  of  success.  The  primary  goals  of  the 
research  component  included  the  professors  learning  the  basics  of  different  educational  research 
designs,  the  importance  of  controlling  and  isolating  the  effects  of  variables,  some  basic  concepts 
of  statistical  analysis,  and  ultimately,  the  implementation  by  each  professor  of  an  actual  research 
project  in  their  own  classroom.  All  of  these  goals  were  met  in  this  program.  In  addition,  the 
professors’  learning  was  enhanced  as  they  evaluated  their  procedures  and  methodologies  - 
normal  activities  for  any  researcher  - and  developed  hypotheses  that  could  explain  the  lack  of 
significant  results.  Finally,  we  hope  that  this  has  begun  a program  of  research  on  the  Scholarship 
of  Teaching  in  which  each  professor  analyzed  his/her  research  results,  considered  and  explained 
the  results,  and  then  will  continue  individual  programs  of  research  by  designing  a follow  up 
study  with  related  or  different  research  question  on  teaching  and  learning.  We  also  hope  that  they 
continue  research  as  a Learning  Community,  where  they  collectively  interact  and  engage 
together  to  design  and  execute  new  studies.  The  specific  research  designs  along  with  the  basic 
statistical  results  are  presented  below.  For  specific  study  results,  see  results  sections  below. 
Please  note  that  two  different  professors  researched  the  same  questions  presented  in  2 and  4 
below.  Also  another  two  different  professors  researched  the  same  questions,  presented  in  5 and  7 
below.  The  research  questions  across  all  Learning  Community  members  were 

1 . Does  individual  performance-based  learning  or  cooperative  performance-based 
learning  result  in  better  student  learning  as  reflected  in  an  end-of-unit  exam? 

2.  a.  Does  a performance  assessment  administered  in  conjunction  with  a traditional  test 
result  in  increased  learning  beyond  the  traditional  test  alone  as  indicated  by  scores  on 
the  traditional  test? 

b.  Does  the  order  of  the  administration  of  a performance  assessment  and  a traditional 
test  result  in  differential  learning  as  indicated  by  the  combination  of  the  performance 
assessment  and  traditional  test  scores? 

c.  Does  the  order  of  administration  of  a performance  assessment  and  a traditional  test 
affect  knowledge  retention  as  indicated  by  a final  exam? 
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3.  a.  In  problem-based  learning  projects,  are  hands-on  manipulative  procedures  more 
effective  on  learning  than  graphical  procedures  as  reflected  in  a midterm  exam? 

b.  In  problem-based  learning  projects,  are  hands-on  manipulative  procedures  more 
effective  on  retention  of  learning  than  graphical  procedures  as  reflected  in  a final  exam? 

4.  a.  Does  a performance  assessment  administered  in  conjunction  with  a traditional  test 
result  in  increased  learning  beyond  the  traditional  test  alone  as  indicated  by  scores  on  the 
traditional  test? 

b.  Does  the  order  of  the  administration  of  a performance  assessment  and  a traditional 
test  result  in  differential  learning  as  indicated  by  the  combination  of  the  performance 
assessment  and  traditional  test  scores? 

c.  Does  the  order  of  administration  of  a performance  assessment  and  a traditional  test 
affect  knowledge  retention  as  indicated  by  a final  exam? 

5.  a.  Does  individual  learning  vs.  cooperative  learning  result  in  differential  knowledge  gains 
as  indicated  by  a traditional  test? 

b.  Does  individual  learning  vs.  cooperative  learning  result  in  differential  knowledge 
retention  as  indicated  by  a final  exam. 

6.  a.  Does  the  administration  of  a performance  assessment  that  covers  a subset  of 
content  also  covered  in  a traditional  midterm  result  in  increased  learning  beyond  the 
administration  of  the  traditional  midterm  alone  as  indicated  by  the  scores  on  the 
traditional  midterm? 

b.  Does  the  administration  of  a performance  assessment  that  covers  a subset  of  content 
also  covered  in  a traditional  midterm  result  in  increased  knowledge  retention  beyond  the 
administration  of  the  traditional  midterm  alone  as  indicated  by  a separate  final  exam? 

7.  a.  Does  individual  learning  vs.  cooperative  learning  result  in  differential  knowledge  gains 
as  indicated  by  a traditional  test? 

b.  Does  individual  learning  vs.  cooperative  learning  result  in  differential  knowledge 
retention  as  indicated  by  a final  exam? 
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Ibrahim  Abdel-Motaleb 


Custom  Model:  Individual  Performance-Based  Learning  vs.  Cooperative  Performance-Based 
Learning 

The  basic  research  question  in  this  model  is: 

1 . Does  individual  performance-based  learning  or  cooperative  performance-based  learning  result  in 
better  student  learning  as  reflected  in  an  end-of-unit  exam  (either  midterm  or  final). 


Table  B.  12.1 


Research  Model:  Individual  vs.  Cooperative  Performance-Based  Learning 


Treatment 

Posttest 

Treatment 

Posttest 

Experimental 
Group  1 

Instruction 

Individual 

Performance- 

Based 

Midterm 

Instruction 

Cooperative 

Performance- 

Based 

Pinal 

Experimental 
Group  2 

Instruction 

Cooperative 

Performance- 

Based 

Midterm 

Instruction 

Individual 

Performance- 

Based 

Pinal 

Replication  1 

Replication  2 

Students  will  be  randomly  assigned  to  the  two  experimental  groups  and,  therefore,  those  groups  can 
be  considered  equivalent.  The  experiment  will  actually  be  administered  twice  during  the  semester. 
To  enhance  the  fairness  to  the  class  both  groups  will  be  exposed  to  both  treatment  conditions  - 
alternating  across  the  two  replications.  The  final  is  not  a comprehensive  final,  so  there  should  be  no 
carryover  from  the  midterm  to  the  final  that  might  contaminate  the  interpretation  of  the  results. 

The  research  question  will  be  addressed  by  comparing  the  mean  test  scores  between  the  two  groups 
for  each  replication  of  the  experiment. 
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Ibrahim  Abdel-Motaleb  - Research  Results 


During  a performance  based  learning  unit  the  24  students  in  the  class  were  assigned  to  either  an 
individual  learning  environment  or  a cooperative  learning  environment.  The  midterm  exam  covered 
the  material  taught  during  the  performance  based  learning  unit. 

The  independent  variable  in  this  context  is  the  learning  environment,  either  individual  or 
cooperative.  The  dependent  variable  is  the  score  on  the  midterm  exam.  The  results  of  the  statistical 
analysis  are  presented  in  the  table  below.  The  statistical  significance  level  is  based  on  an 
independent  samples  t-test. 


Table  B.  12.2 


Variable 

Group 

N 

Mean 

SD 

Sig.  Level 
(df=22) 

1 . Midterm 

Individual 

9 

22.4 

3.5 

.005 

Cooperative 

15 

28.7 

6.1 

Research  Question: 

1 . The  difference  between  means  for  the  individual  learning  environment  and  the  cooperative 
learning  environment  from  the  midterm  was  statistically  significant  beyond  the  .05  level  of 
significance. 

This  statistical  significance  indicates  that  a null  hypothesis  specifying  equal  effects  of  individual 
learning  and  cooperative  learning  in  this  context  can  be  rejected.  The  cooperative  learning 
environment  appears  to  have  produced  more  learning,  based  on  the  midterm  exam  scores,  than  the 
individual  learning  environment. 

[The  original  design  for  this  experiment  specified  two  replications  - one  from  the  midterm  and  one 
from  the  final  exam.  However,  data  from  the  second  replication  were  not  submitted  for  analysis. 
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Ibrahim  Abdel-Motaleb  - SPSS  Output 


Table  B.  12.3 


Group  Statistics 


Group 

Assignment 

N 

Mean 

Std. 

Deviation 

Std.  Error 
Mean 

Midterm  Score 

individuai 

9 

22.4444 

3.53946 

1.17982 

Cooperative 

15 

28.6667 

6.16055 

1 .59065 

Table  B.  12.4 


Independent  Samples  Test 


Levene's  Test 
fer  Equaiity  of 
Variances 

t-test  for  Equaiity  of  Means 

F 

Sig. 

t 

df 

Sig.  (2- 
taiied) 

Mean 

Difference 

Std.  Errer 
Difference 

95%  Confidence  Intervai 
of  the  Difference 

Lewer 

Upper 

Midterm  Scere 

Equai 

variances 

assumed 

6.369 

.019 

-2.754 

22 

.012 

-6.22222 

2.25909 

-10.90728 

-1.53716 

Equai 
variances 
not  assumed 

-3.142 

21.993 

.005 

-6.22222 

1.98044 

-10.32948 

-2.11497 
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Abul  Azad 


Performance  Assessment  and  Traditional  Test  Administered  in  Different  Order 


This  study  is  designed  to  examine  the  following  questions: 

1.  Does  a performance  assessment  administered  in  conjunction  with  a traditional  test  result  in 
increased  learning  beyond  the  traditional  test  alone  as  indicated  by  scores  on  the  traditional  test? 

2.  Does  the  order  of  the  administration  of  a performance  assessment  and  a traditional  test  result  in 
differential  learning  as  indicated  by  the  combination  of  the  performance  assessment  and 
traditional  test  scores? 

3.  Does  the  order  of  administration  of  a performance  assessment  and  a traditional  test  affect 
knowledge  retention  as  indicated  by  a final  exam? 

The  basic  design  for  this  study  is  presented  in  the  table  below. 

Table  B.  12.5 

Model  2:  Performance  Assessment  and  Traditional  Test 
Administered  in  Different  Order 


Group  1 

Instruction 

Performance 

Assessment 

Traditional  Test 

Final 

Group  2 

Instruction 

Traditional  Test 

Performance 

Assessment 

-> 

Final 

Students  will  be  randomly  assigned  to  the  experimental  and  control  groups  and  therefore  those 
groups  can  be  considered  equivalent.  Group  1 will  receive  the  performance  assessment  before  the 
traditional  test  and  group  2 will  receive  the  traditional  test  before  the  performance  assessment. 

Question  1 will  be  addressed  by  comparing  the  traditional  test  means  between  the  two  groups. 

Question  2 will  be  addressed  by  comparing  the  combined  performance  assessment  and  traditional 
test  scores  between  the  two  groups. 

Question  3 will  be  addressed  by  comparing  scores  from  the  final  that  are  based  on  material  also 
covered  on  the  midterm  (the  performance  assessment  and  the  traditional  midterm  test)  between  the 
two  groups. 
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Abul  Azad  - Research  Results 


The  fifteen  students  in  the  class  were  divided  into  the  two  treatment  groups;  group  1 was  comprised  of 
eight  students  who  took  the  midterm  performance  assessment  followed  by  the  traditional  midterm  and 
group  2 was  comprised  of  seven  students  who  took  the  traditional  midterm  followed  by  the  performance 
assessment. 

The  independent  variable  in  this  context  is  group  assignment.  The  dependent  variables  depend  on  the 
specific  research  questions  addressed.  The  results  of  the  statistical  analysis  for  each  dependent  variable  are 
presented  in  the  table  below.  The  statistical  significance  levels  are  based  on  independent  samples  t-tests. 


Table  B.  12.6 


Variable 

Group 

N 

Mean 

SD 

Sig.  Level 
(df=13) 

1 . Traditional  Midterm 

1.  Perf.  First 

8 

91.1 

7.3 

.69 

2.  Trad.  First 

7 

92.4 

4.4 

2.  Combined  Traditional  Midterm  & Performance 
Assessment 

1.  Perf.  First 

8 

162.3 

33.5 

.55 

2.  Trad.  First 

7 

171.4 

23.4 

3.  Final  Exam  - Midterm  Content  Only 

1.  Perf.  First 

8 

22.9 

13.9 

.16 

2.  Trad.  First 

7 

30.9 

3.1 

Research  Questions: 

1 . The  difference  between  group  means  for  variable  1 - the  traditional  midterm  exam,  was  not  statistically 
significant  beyond  the  .05  level  of  significance. 

2.  The  difference  between  the  group  means  for  variable  2 - the  combined  traditional  midterm  exam  and  the 
midterm  performance  assessment,  was  not  statistically  significant. 

3.  The  difference  between  the  group  means  for  variable  3 - the  final  exam  scores  for  the  midterm  content, 
was  not  statistically  significant. 

Although,  for  all  three  variables,  the  mean  for  students  taking  the  traditional  midterm  first  were  higher  than 
the  mean  for  students  taking  the  performance  test  first,  none  of  the  mean  differences  was  statistically 
significant.  This  lack  of  statistical  significance  indicates  that  a null  hypothesis  specifying  equal  effects  of 
treatment  groups  in  this  context  is  still  a viable  hypothesis  and  cannot  be  rejected.  Some  possible  technical 
reasons  for  this  result  could  relate  to  the  small  sample  sizes  and,  in  some  cases,  the  relatively  large  standard 
deviations,  both  of  which  can  be  related  to  non- significant  results.  There  are  also  possible  classroom 
context  and  operational  factors  which  could  be  related  to  the  results.  These  factors  should  be  considered 
and  discussed  by  the  experimenter. 

Technical  note:  Although  some  of  the  group  variances  above  are  clearly  not  equal  within  each  variable, 
SPSS  performs  the  analyses  based  on  both  equal  and  non-equal  group  variances.  In  either  case,  the 
differences  between  means  were  not  significant. 
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Table  B.  12.7 


Group  Statistics 


Group  Assignment 

N 

Mean 

Std. 

Deviation 

Std.  Error 
Mean 

Midterm  Traditional 
Assessment 

Performance  First 

8 

91.1250 

7.25923 

2.56653 

Traditional  First 

7 

92.4286 

4.42934 

1.67413 

Combined  Trad  & 
PA 

Performance  First 

8 

162.2500 

33.47387 

11.83480 

Traditional  First 

7 

171.4286 

23.35135 

8.82598 

Final  on  Midterm 
Content 

Performance  First 

8 

22.8750 

13.93287 

4.92602 

Traditional  First 

7 

30.8571 

3.13202 

1.18379 

Traditional  First 

7 

79.0000 

24.27619 

9.17554 

Table  B.  12.8 


Independent  Samples  Test 


Levene's  Test 
for  Equality  of 
Variances 

t-test  for  Equality  of  Means 

F 

Sig. 

t 

df 

Sig.  (2- 
tailed) 

Mean 

Difference 

Std.  Error 
Difference 

95%  Confidence  Interval 
of  the  Difference 

Lower 

Upper 

Midterm 

Traditional 

Assessment 

Equal 

variances 

assumed 

1.058 

.322 

-.412 

13 

.687 

-1.30357 

3.16637 

-8.14409 

5.53695 

Equal 

variances  not 
assumed 

-.425 

1 1 .744 

.678 

-1.30357 

3.06427 

-7.99625 

5.38910 

Combined  Trad  & 
PA 

Equal 

variances 

assumed 

3.346 

.090 

-.607 

13 

.555 

-9.17857 

15.13348 

-41 .87247 

23.51533 

Equal 

variances  not 
assumed 

-.622 

12.456 

.545 

-9.17857 

14.76348 

-41.21521 

22.85807 

Final  on  Midterm 
Content 

Equal 

variances 

assumed 

8.352 

.013 

-1.477 

13 

.164 

-7.98214 

5.40477 

-19.65843 

3.69415 

Equal 

variances  not 
assumed 

-1.576 

7.802 

.155 

-7.98214 

5.06626 

-19.71690 

3.75262 
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Brianno  Coller 


Model  1:  Hands-On  Manipulative  Procedures  vs.  Graphical  Procedures 
This  study  is  designed  to  examine  two  questions: 

1 . In  problem-based-leaming  projects,  are  hands-on  manipulative  procedures  more  effective  on 
learning  than  graphical  procedures  as  reflected  in  a midterm  exam? 

2.  In  problem-based-leaming  projects,  are  hands-on  manipulative  procedures  more  effective  on 
retention  of  learning  than  graphical  procedures  as  reflected  in  a final  exam? 

Table  B.  12.9 


Model  1:  Hands-On  Manipulative  vs.  Graphical 


Treatment 

Posttest  1 

Posttest  2 

Experimental 
Group  1 

Instruction 

Hands-On 

Manipulative 

Midterm 

^ 

Pinal 

Experimental 
Group  2 

Instruction 

Graphical 

Midterm 

Pinal 

Students  will  be  randomly  assigned  to  the  two  experimental  groups  and,  therefore,  those  groups  can 
be  considered  equivalent.  Each  group  will  be  given  some  problem-based  learning  tasks;  group  1 
will  attempt  to  resolve/complete  the  tasks  with  hands-on  manipulation  of  physical  objects,  while 
group  2 will  attempt  to  resolve/complete  the  problems  with  graphical  techniques.  After  the 
administration  of  the  midterm  (posttest  1),  the  distinction  between  the  groups  is  dissolved  with 
instruction  and  assessment  activities  delivered  to  all  students  equally. 

Question  1 will  be  addressed  by  comparing  the  posttest  1 means  between  the  two  groups.  Question 
2 will  be  addressed  by  comparing  the  posttest  2 means  between  the  groups  and  by  comparing  the 
posttest  1 means  to  the  posttest  2 means  (for  the  items  in  the  final  that  cover  material  in  the 
midterm)  across  the  two  groups. 
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Brianno  Coller  - Research  Results 


Sixty-four  students  in  the  class  were  assigned  to  either  Group  1 using  a Hands-On  Manipulative 
procedure  or  to  Group  2 using  a Graphical  procedure  while  working  on  problem-based  learning 
projects.  All  students  were  given  a midterm  exam  on  concepts  covered;  the  final  exam  also  covered 
some  of  the  same  concepts  covered  by  the  midterm. 

The  independent  variable  in  this  context  is  group  assignment,  either  hands-on  manipulative  or 
graphical.  The  dependent  variable  is  either  the  midterm  score  or  the  score  on  the  portion  of  the  final 
covering  midterm  content,  depending  on  the  research  question.  The  results  of  the  statistical  analysis 
are  presented  in  the  table  below.  The  statistical  significance  level  is  based  on  independent  samples 
t- tests. 


Table  B.  12. 10 


Variable 

Group 

N 

Mean 

SD 

Sig.  Level 
(df=62) 

1.  Midterm 

Hands  On 
Manipulative 

30 

19.6 

5.4 

.96 

Graphical 

34 

19.7 

5.5 

2.  Final  Exam  - Midterm  Content  Only 

Hands  On 
Manipulative 

30 

63.2 

16.6 

.77 

Graphical 

34 

64.6 

20.9 

Research  Questions: 

1.  The  difference  between  means  for  the  hands-on  manipulative  group  and  the  graphical  group  for 
the  midterm  was  not  statistically  significant  beyond  the  .05  level  of  significance. 

2.  The  difference  between  means  for  the  hands-on  manipulative  group  and  the  graphical  group  for 
the  final  exam  content  that  was  also  covered  on  the  midterm  was  not  statistically  significant  beyond 
the  .05  level  of  significance. 

This  lack  of  statistical  significance  indicates  that  a null  hypothesis  specifying  equal  effects  of  hands-on 
manipulative  and  graphical  procedures  in  this  context  is  still  a viable  hypothesis  and  cannot  be  rejected. 
Some  possible  reasons  for  this  result  could  be  related  to  classroom  context  and  operational  factors.  These 
factors  should  be  considered  and  discussed  by  the  experimenter. 
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Table  B.  12. 11 


Group  Statistics 


Group  Assignment 

N 

Mean 

Std. 

Deviation 

Std.  Error 
Mean 

Midterm 

Hands-On 

Manipulative 

30 

19.60 

5.437 

.993 

Graphical 

34 

19.68 

5.465 

.937 

Final  - MT  Content 
(Zeros  Deleted) 

Hands-On 

Manipulative 

30 

63.23 

16.565 

3.024 

Graphical 

34 

64.65 

20.875 

3.580 

Table  B.  12. 12 


Independent  Samples  Test 


Levene's  Test 
for  Equality  of 
Variances 

t-test  for  Equality  of  Means 

F 

Siq. 

t 

df 

Sig.  (2- 
tailed) 

Mean 

Difference 

Std.  Error 
Difference 

95%  Confidence 
Interval  of  the 
Difference 

Lower 

Midterm 

Equal  variances 
assumed 

.128 

.721 

-.056 

62 

.956 

-.076 

1.366 

-2.806 

2.653 

Equal  variances 
not  assumed 

-.056 

61.088 

.956 

-.076 

1.365 

-2.806 

2.653 

Final  - MT 
Content  (Zeros 
Deleted) 

Equal  variances 
assumed 

1.148 

.288 

-.297 

62 

.767 

-1.414 

4.755 

-10.918 

8.090 

Equal  variances 
not  assumed 

-.302 

61.352 

.764 

-1.414 

4.686 

-10.784 

7.956 
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Abhijit  Gupta 

Performance  Assessment  and  Traditional  Test  Administered  in  Different  Order 


This  study  is  designed  to  examine  the  following  questions: 

1.  Does  a performance  assessment  administered  in  conjunction  with  a traditional  test  result  in 
increased  learning  beyond  the  traditional  test  alone  as  indicated  by  scores  on  the  traditional  test? 

2.  Does  the  order  of  the  administration  of  a performance  assessment  and  a traditional  test  result  in 
differential  learning  as  indicated  by  the  combination  of  the  performance  assessment  and 
traditional  test  scores? 

3.  Does  the  order  of  administration  of  a performance  assessment  and  a traditional  test  affect 
knowledge  retention  as  indicated  by  a final  exam? 

The  basic  design  for  this  study  is  presented  in  the  table  below. 

Table  B.  12. 13 

Model  2:  Performance  Assessment  and  Traditional  Assessment 
Administered  in  Different  Order 


Group  1 

Instruction 

Performance 

Assessment 

Traditional  Test 

Final 

Group  2 

Instruction 

Traditional  Test 

Performance 

Assessment 

-> 

Final 

Students  will  be  randomly  assigned  to  the  experimental  and  control  groups  and  therefore  those 
groups  can  be  considered  equivalent.  Group  1 will  receive  the  Performance  assessment  before  the 
traditional  test  and  group  2 will  receive  the  traditional  test  before  the  Performance  assessment. 

Question  1 will  be  addressed  by  comparing  the  traditional  test  means  between  the  two  groups. 

Question  2 will  be  addressed  by  comparing  the  combined  performance  assessment  and  traditional 
test  scores  between  the  two  groups. 

Question  3 will  be  addressed  by  comparing  scores  from  the  final  that  are  based  on  material  also 
covered  on  the  midterm  (the  performance  assessment  and  the  traditional  midterm  test)  between  the 
two  groups. 
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Abhijit  Gupta  - Research  Results 


The  44  students  in  the  class  were  divided  into  the  two  treatment  groups:  group  1 was  comprised  of  22 
students  who  took  the  midterm  performance  assessment  followed  by  the  traditional  midterm,  and  group  2 
was  comprised  of  22  students  who  took  the  traditional  midterm  followed  by  the  performance  assessment. 

The  independent  variable  in  this  context  is  group  assignment.  The  dependent  variables  depend  on  the 
specific  research  questions  addressed.  The  results  of  the  statistical  analysis  for  each  dependent  variable  are 
presented  in  the  table  below.  The  statistical  significance  levels  are  based  on  independent  samples  t-tests. 


Table  B.  12. 14 


Variable 

Group 

N 

Mean 

SD 

Sig.  Level 
(df=42) 

1 . Traditional  Midterm 

1.  Perf.  First 

22 

41.0 

8.9 

.97 

2.  Trad.  First 

22 

41.2 

14.2 

2.  Combined  Traditional  Midterm  & Performance 
Assessment 

1.  Perf.  First 

22 

66.8 

9.8 

.53 

2.  Trad.  First 

22 

69.1 

14.4 

3.  Final  Exam  - Midterm  Content  Only 

1.  Perf.  First 

22 

25.7 

6.4 

.13 

2.  Trad.  First 

22 

29.0 

7.8 

Research  Questions: 

1 . The  difference  between  group  means  for  variable  1 - the  traditional  midterm  exam,  was  not  statistically 
significant  beyond  the  .05  level  of  significance. 

2.  The  difference  between  the  group  means  for  variable  2 - the  combined  traditional  midterm  exam  and  the 
midterm  performance  assessment,  was  not  statistically  significant. 

3.  The  difference  between  the  group  means  for  variable  3 - the  final  exam  scores  for  the  midterm  content, 
was  not  statistically  significant. 

Although,  for  all  three  variables,  the  mean  for  students  taking  the  traditional  midterm  first  were  higher  than 
the  mean  for  students  taking  the  performance  test  first,  none  of  the  mean  differences  was  statistically 
significant.  This  lack  of  statistical  significance  indicates  that  a null  hypothesis  specifying  equal  effects  of 
treatment  groups  in  this  context  is  still  a viable  hypothesis  and  cannot  be  rejected.  Some  possible  reasons 
for  this  result  could  be  related  to  classroom  context  and  operational  factors.  These  factors  should  be 
considered  and  discussed  by  the  experimenter. 

Technical  note:  Although  some  of  the  group  variances  above  are  clearly  not  equal  within  each  variable, 
SPSS  performs  the  analyses  based  on  both  equal  and  non-equal  group  variances.  In  either  case,  the 
differences  between  means  were  not  significant. 
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Table  B.  12. 15 


Group  Statistics 


Group  Assignment 

N 

Mean 

Std. 

Deviation 

Std.  Error 
Mean 

Midterm  Traditional 
Assessment 

Performance  First 

22 

41.0455 

8.93083 

1.90406 

Traditional  First 

22 

41.1818 

14.22820 

3.03346 

Combined  Trad  & PA 

Performance  First 

22 

66.7727 

9.76355 

2.08160 

Traditional  First 

22 

69.1364 

14.40666 

3.07151 

Final  on  Midterm  Content 

Performance  First 

22 

25.6818 

6.42455 

1.36972 

Traditional  First 

22 

29.0000 

7.84978 

1.67358 

Table  B.  12. 16 


Independent  Samples  Test 


Levene's  Test 
for  Equality  of 
Variances 

t-test  for  Equality  of  Means 

F 

Sig. 

t 

df 

Sig.  (2- 
tailed) 

Mean 

Difference 

Std.  Error 
Difference 

95%  Confidence 
Interval  of  the 
Difference 

Lower 

Upper 

Midterm  Traditional 
Assessment 

Equal 

variances 

assumed 

2.323 

.135 

-.038 

42 

.970 

-.13636 

3.58153 

-7.36418 

7.09145 

Equal 

variances  not 
assumed 

-.038 

35.32 

.970 

-.13636 

3.58153 

-7.40487 

7.13214 

Equal 

variances 

assumed 

1.293 

.262 

-.637 

42 

.528 

-2.36364 

3.71042 

-9.85157 

5.12429 

Combined  Trad  & PA 

Equal 

variances  not 
assumed 

-.637 

36.93 

.528 

-2.36364 

3.71042 

-9.88214 

5.15487 

Equal 

variances 

assumed 

-1.534 

42 

.132 

-3.31818 

2.16264 

-7.68256 

1 .04620 

Final  on  Midterm 
Content 

Equal 

variances  not 
assumed 

1.653 

.206 

-1.534 

40.42 

.133 

-3.31818 

2.16264 

-7.68762 

1.05126 
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Model  4 - Individual  Learning  vs.  Cooperative  Learning 
This  study  is  designed  to  examine  the  following  questions: 

1.  Does  individual  learning  vs.  cooperative  learning  result  in  differential  knowledge  gains  as 
indicated  by  a traditional  test? 

2.  Does  individual  learning  vs.  cooperative  learning  result  in  differential  knowledge  retention  as 
indicated  by  a final  exam? 

The  design  for  this  study  is  presented  in  the  table  below. 


Table  B.  12. 17 


Model  4:  Individual  Learning  vs.  Cooperative  Learning 


Treatment 

Posttest  1 

Posttest  2 

Individual 

Learning 

Group 

Individual 

Learning 

Traditional  Test 

Final 

Cooperative 

Learning 

Group 

Cooperative 

Learning 

Traditional  Test 

Final 

Treatment 

Group 

Content  Area 
I 

Content  Area 
II 

Content  Area 
III 

Content  Area 
IV 

1 

Individual 

Cooperative 

Individual 

Cooperative 

2 

Cooperative 

Individual 

Cooperative 

Individual 

Random  assignment  of  students  to  two  groups  will  allow  us  to  assume  the  groups  are  equivalent. 
The  actual  delivery  of  the  treatment  conditions  will  alternate  across  content  areas  and  groups,  as 
shown  in  the  blowout  diagram;  group  1 will  be  the  individual  learning  group  for  content  areas  I & 
III,  while  group  2 will  be  the  individual  learning  group  for  content  areas  II  & IV.  This  will  add  to 
the  validity  of  the  design  and  enhance  the  fairness  of  the  treatment  conditions  within  the  student 
groups.  For  this  delivery  model  to  work  there  needs  to  be  an  even  number  of  content  areas  with  a 
minimum  of  two.  For  fairness  to  students,  each  content  area  should  also  be  weighted  approximately 
equally  within  both  posttests.  After  the  administration  of  the  Traditional  Test  (Posttest  1)  the 
distinction  between  the  groups  is  dissolved  with  instruction  and  assessment  activities  delivered  to 
all  students  equally. 

Question  1 will  be  addressed  by  comparing  the  posttest  1 means  under  the  individual  and 
cooperative  learning  conditions.  Question  2 will  be  addressed  with  similar  comparisons  on  the 
posttest  2 means. 
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Reinaldo  Moraga  - Research  Results 


The  18  students  in  the  class  were  assigned  to  either  Group  1 or  to  Group  2,  with  9 students  in  each 
group.  Then  four  different  areas  of  content  were  taught  to  all  18  students.  For  content  area  I students 
in  group  1 worked  individually  and  students  in  group  2 worked  cooperatively  (in  small  groups).  For 
content  area  II,  the  individual  and  cooperative  approach  was  reversed  between  the  two  groups.  This 
was  repeated,  with  the  reversals,  for  content  areas  III  and  IV.  The  content  was  covered  on  the 
midterm  and  some  of  the  same  content  was  covered  on  the  final.  Thus  for  both  the  midterm  and 
final,  each  student  obtained  a score  for  content  he  or  she  learned  on  an  individual  basis  and  a score 
for  content  he  or  she  learned  on  a cooperative  basis. 

The  independent  variable  in  this  context  is  learning  environment,  either  individual  or  cooperative.  For 
research  question  1 the  dependent  variable  is  the  score  on  the  midterm  exam  that  matches  individual  or 
cooperative  learning:  that  is,  a student  obtained  an  individual-learning  score  based  on  the  midterm  content 
that  the  student  learned  while  working  in  the  individual  learning  environment  and  the  student  also  obtained 
a cooperative-learning  score  based  on  the  midterm  content  that  the  student  learned  while  working  in  the 
cooperative  learning  environment.  For  research  question  2,  the  dependent  variable  is  the  score  on  the  final 
exam  that  matches  individual  or  cooperative  learning  for  the  content  on  the  final  that  was  also  covered  on 
the  midterm.  The  results  of  the  statistical  analysis  for  each  dependent  variable  are  presented  in  the  table 
below.  The  statistical  significance  levels  are  based  on  paired  samples  t- tests. 


Table  B. 12.18 


Variable 

Group 

N 

Mean 

SD 

Sig.  Level 
(df=17) 

1 . Midterm 

Individual 

18 

29.1 

8.0 

.08 

Cooperative 

18 

25.8 

9.1 

2.  Final  Exam  - Midterm  Content  Only 

Individual 

18 

3.4 

1.4 

.71 

Cooperative 

18 

3.3 

1.7 

Research  Questions: 

1 . The  difference  between  means  for  the  individual  learning  environment  and  the  cooperative 
learning  environment  for  the  midterm  was  not  statistically  significant  beyond  the  .05  level  of 
significance. 

2.  The  difference  between  means  for  the  individual  learning  environment  and  the  cooperative 
learning  environment  for  the  final  exam  content  that  was  also  covered  on  the  midterm  was  not 
statistically  significant  beyond  the  .05  level  of  significance. 

This  lack  of  statistical  significance  indicates  that  a null  hypothesis  specifying  equal  effects  of  individual 
learning  and  cooperative  learning  in  this  context  is  still  a viable  hypothesis  and  cannot  be  rejected.  Some 
possible  reasons  for  this  result  could  be  related  to  classroom  context  and  operational  factors.  These  factors 
should  be  considered  and  discussed  by  the  experimenter. 
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Table  B.  12. 19 


Paired  Samples  Statistics 


Mean 

N 

Std. 

Deviation 

Std.  Error 
Mean 

Pair  1 

Midterm  Individual 

29.1111 

18 

7.97709 

1 .88022 

Midterm  Cooperative 

25.7778 

18 

9.12370 

2.15048 

Pair  2 

Final  MT  Content 
Individual 

3.4444 

18 

1.38148 

.32562 

Final  MT  Content 
Cooperative 

3.3333 

18 

1 .68034 

.39606 

Table  B.  12.20 


Paired  Samples  Correlations 


N 

Correlation 

Sig. 

Pair  1 

Midterm  Individual  & 
Midterm  Cooperative 

18 

.605 

.008 

Pair  2 

Final  MT  Content 
Individual  & Final  MT 
Content  Cooperative 

18 

.693 

.001 

Table  B.  12.21 


Paired  Samples  Test 


Paired  Differenoes 

t 

df 

Sig.  (2- 
tailed) 

Mean 

Std. 

Deviation 

Std.  Error 
Mean 

95%  Confidence  Interval 
of  the  Difference 

Lower 

Upper 

Pair  1 

Midterm  Individual  - 
Midterm  Cooperative 

3.33333 

7.66965 

1 .80775 

-.48069 

7.14736 

1.844 

17 

.083 

Pair  2 

Final  MT  Content 
Individual  - Final  MT 
Content  Cooperative 

.11111 

1.23140 

.29024 

-.50125 

.72347 

.383 

17 

.707 

18 


Regina  Rahn 

Model  1 -Traditional  Test  With  Performance  Assessment 


This  study  is  designed  to  examine  the  following  questions: 

1.  Does  the  administration  of  a performance  assessment  that  covers  a subset  of  content  also  covered 
in  a traditional  midterm  result  in  increased  learning  beyond  the  administration  of  the  traditional 
midterm  alone  as  indicated  by  the  scores  on  the  traditional  midterm? 

2.  Does  the  administration  of  a performance  assessment  that  covers  a subset  of  content  also  covered 
in  a traditional  midterm  result  in  increased  knowledge  retention  beyond  the  administration  of  the 
traditional  midterm  alone  as  indicated  by  a separate  final  exam? 

The  basic  design  for  this  study  is  presented  in  the  table  below. 

Table  B.  12.22 


Model  1:  Traditional  Test  With  Performance  Assessment 


Treatment 

Posttest  1 

Posttest  2 

Experimental 

Group 

Instruction 

Performance 
Assessment 
Related  to 
Traditional  Test 

Traditional 

Midterm 

Traditional  Final 

Control 

Group 

Instruction 

Performance 
Assessment  Not 
Related  to 
Traditional  Test 
(Placebo) 

Traditional 

Midterm 

-> 

Traditional  Final 

Students  will  be  randomly  assigned  to  the  experimental  and  control  groups  and  therefore  those 
groups  can  be  considered  equivalent.  The  treatment  for  the  experimental  group  is  essentially  the 
administration  of  the  traditional  midterm  along  with  some  performance  assessment  activities 
covering  some  of  the  same  content  that  is  covered  in  the  traditional  midterm.  The  treatment  for  the 
control  group  is  the  administration  of  the  traditional  midterm  and  a placebo  - perhaps  some 
performance  assessment  activities  that  are  not  related  to  the  content  in  the  traditional  midterm. 
After  the  administration  of  the  Traditional  Test  (Posttest  1),  the  distinction  between  the  groups  is 
dissolved  with  instruction  and  assessment  activities  delivered  to  all  students  equally. 

Question  1 will  be  addressed  by  comparing  the  traditional  midterm  means  across  the  experimental 
and  control  groups.  Question  2 will  be  addressed  by  comparing  the  final  exam  means  across  the 
experimental  and  control  groups  (for  the  content  in  the  final  that  is  also  in  the  midterm)  across  the 
two  groups. 
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Regina  Rahn  - Research  Results 


The  14  students  in  the  class  were  assigned  to  either  Group  1,  the  Decision  Tree  environment,  or  to 
Group  2,  the  Fault  Tree  environment.  The  independent  variable  in  this  context  is  learning 
environment,  either  decision  tree  or  fault  tree.  The  dependent  variable  is  the  score  on  the  midterm 
for  research  question  1 and  the  score  on  the  portion  of  the  final  exam  over  material  that  was  covered 
on  the  midterm  for  research  question  2. 

The  results  of  the  statistical  analysis  for  each  dependent  variable  are  presented  in  the  table  below.  The 
statistical  significance  levels  are  based  on  independent  samples  t- tests. 


Table  B.  12.23 


Variable 

Group 

N 

Mean 

SD 

Sig.  Level 
(df=12) 

1 . Midterm 

Decision  Tree 

7 

74.7 

7.6 

.27 

Fault  Tree 

7 

79.7 

8.8 

2.  Final  Exam  - Midterm  Content  Only 

Decision  Tree 

7 

25.1 

4.1 

.19 

Fault  Tree 

7 

22.4 

3.0 

Research  Questions: 

1 . The  difference  between  means  for  the  decision  tree  environment  and  the  fault  tree  environment 
for  the  midterm  was  not  statistically  significant  beyond  the  .05  level  of  significance. 

2.  The  difference  between  means  for  the  decision  tree  environment  and  the  fault  tree  environment 
for  the  final  exam  content  that  was  also  covered  on  the  midterm  was  not  statistically  significant 
beyond  the  .05  level  of  significance. 

This  lack  of  statistical  significance  indicates  that  a null  hypothesis  specifying  equal  effects  of  decision  tree 
and  fault  tree  learning  environments  in  this  context  is  still  a viable  hypothesis  and  cannot  be  rejected. 

Some  possible  reasons  for  this  result  could  be  related  to  classroom  context  and  operational  factors.  These 
factors  should  be  considered  and  discussed  by  the  experimenter. 
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Regina  Rahn  - SPSS  Output 


Table  B.  12.24 


Groui 

p Statistics 

Group 

Assignment 

N 

Mean 

Std. 

Deviation 

Std.  Error 
Mean 

Midterm  DT  Content 

DT  Group 

7 

18.2857 

3.54562 

1.34012 

FT  Group 

7 

16.8571 

4.52506 

1.71031 

Midterm  Totai 

DT  Group 

7 

74.7143 

7.56559 

2.85952 

FT  Group 

7 

79.7143 

8.82637 

3.33605 

Finai  DT  Content 

DT  Group 

7 

25.1429 

4.09994 

1 .54963 

FT  Group 

7 

22.4286 

3.04725 

1.15175 

Finai  FT  Content 

DT  Group 

7 

35.7143 

2.98408 

1.12788 

FT  Group 

7 

38.4286 

2.43975 

.92214 

Finai  Totai 

DT  Group 

7 

84.0000 

7.83156 

2.96005 

FT  Group 

7 

81.2857 

7.65320 

2.89264 

Table  B.  12.25 


Independent  Samples  Test 


Levene's  Test 
for  Equaiity  of 
Variances 

t-test  for  Equaiity  of  Means 

F 

Siq. 

t 

df 

Sig.  (2- 
taiied) 

Mean 

Difference 

Std.  Error 
Difference 

95%  Confidence  Intervai 
of  the  Difference 

Lower 

Upper 

Midterm  DT 
Content 

Equai 

variances 

assumed 

1.938 

.189 

.657 

12 

.523 

1.42857 

2.17281 

-3.30557 

6.16271 

Equai 
variances 
not  assumed 

.657 

11.351 

.524 

1.42857 

2.17281 

-3.33578 

6.19293 

Midterm 

Totai 

Equai 

variances 

assumed 

.607 

.451 

-1.138 

12 

.277 

-5.00000 

4.39387 

-14.57343 

4.57343 

Equai 
variances 
not  assumed 

-1.138 

11.726 

.278 

-5.00000 

4.39387 

-14.59832 

4.59832 

Finai  DT 
Content 

Equai 

variances 

assumed 

2.024 

.180 

1.406 

12 

.185 

2.71429 

1.93077 

-1.49251 

6.92108 

Equai 
variances 
not  assumed 

1.406 

11.079 

.187 

2.71429 

1.93077 

-1.53163 

6.96020 

Finai  FT 
Content 

Equai 

variances 

assumed 

1.320 

.273 

-1.863 

12 

.087 

-2.71429 

1.45686 

-5.88852 

.45995 

Equai 
variances 
not  assumed 

-1.863 

1 1 .544 

.088 

-2.71429 

1.45686 

-5.90247 

.47390 

Finai  Totai 

Equai 

variances 

assumed 

.017 

.897 

.656 

12 

.524 

2.71429 

4.13875 

-6.30328 

11.73185 

Equai 
variances 
not  assumed 

.656 

11.994 

.524 

2.71429 

4.13875 

-6.30381 

1 1 .73238 
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Model  4 - Individual  Learning  vs.  Cooperative  Learning 
This  study  is  designed  to  examine  the  following  questions: 

1.  Does  individual  learning  vs.  eooperative  learning  result  in  differential  knowledge  gains  as 
indie  ated  by  a traditional  test? 

2.  Does  individual  learning  vs.  eooperative  learning  result  in  differential  knowledge  retention  as 
indieated  by  a final  exam? 

The  design  for  this  study  is  presented  in  the  table  below. 


Table  B.  12.26 


Model  4:  Individual  Learning  vs.  Cooperative  Learning 


Treatment 

Posttest  1 

Posttest  2 

Individual 

Learning 

Group 

Individual 

Learning 

Traditional  Test 

Final 

Cooperative 

Learning 

Group 

Cooperative 

Learning 

Traditional  Test 

Final 

Treatment 

Group 

Content  Area 
I 

Content  Area 
II 

Content  Area 
III 

Content  Area 
IV 

1 

Individual 

Cooperative 

Individual 

Cooperative 

2 

Cooperative 

Individual 

Cooperative 

Individual 

Random  assignment  of  students  to  two  groups  will  allow  us  to  assume  the  groups  are  equivalent. 
The  aetual  delivery  of  the  treatment  eonditions  will  alternate  aeross  eontent  areas  and  groups,  as 
shown  in  the  blowout  diagram.  Group  1 will  be  the  individual  learning  group  for  eontent  areas  I & 
III,  while  group  2 will  be  the  individual  learning  group  for  eontent  areas  II  & IV.  This  will  add  to 
the  validity  of  the  design  and  enhance  the  fairness  of  the  treatment  conditions  within  the  student 
groups.  For  this  delivery  model  to  work  there  needs  to  be  an  even  number  of  content  areas  with  a 
minimum  of  two.  For  fairness  to  students,  each  content  area  should  also  be  weighted  approximately 
equally  within  both  posttests.  After  the  administration  of  the  Traditional  Test  (Posttest  1)  the 
distinction  between  the  groups  is  dissolved  with  instruction  and  assessment  activities  delivered  to 
all  students  equally. 

Question  1 will  be  addressed  by  comparing  the  posttest  1 means  under  the  individual  and 
cooperative  learning  conditions.  Question  2 will  be  addressed  with  similar  comparisons  on  the 
posttest  2 means. 
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Robert  Tatara  - Research  Results 


The  28  students  in  the  class  were  assigned  to  either  Group  1 or  to  Group  2,  with  14  students  in  each 
group.  Then  four  different  areas  of  content  were  taught  to  all  28  students.  For  content  area  I students 
in  group  1 worked  individually  and  students  in  group  2 worked  cooperatively  (in  small  groups).  For 
content  area  II,  the  individual  and  cooperative  approach  was  reversed  between  the  two  groups.  This 
was  repeated,  with  the  reversals,  for  content  areas  III  and  IV.  The  content  was  covered  on  the 
midterm  and  some  of  the  same  content  was  covered  on  the  final.  Thus  for  both  the  midterm  and 
final  each  student  obtained  a score  for  content  he  or  she  learned  on  an  individual  basis  and  a score 
for  content  he  or  she  learned  on  a cooperative  basis. 

The  independent  variable  in  this  context  is  learning  environment,  either  individual  or  cooperative.  For 
research  question  1,  the  dependent  variable  is  the  score  on  the  midterm  exam  that  matches  individual  or 
cooperative  learning.  That  is,  a student  obtained  an  individual-learning  score  based  on  the  midterm  content 
that  the  student  learned  while  working  in  the  individual  learning  environment  and  the  student  also  obtained 
a cooperative-learning  score  based  on  the  midterm  content  that  the  student  learned  while  working  in  the 
cooperative  learning  environment.  For  research  question  2,  the  dependent  variable  is  the  score  on  the  final 
exam  that  matches  individual  or  cooperative  learning  for  the  content  on  the  final  that  was  also  covered  on 
the  midterm.  The  results  of  the  statistical  analysis  for  each  dependent  variable  are  presented  in  the  table 
below.  The  statistical  significance  levels  are  based  on  paired  samples  t- tests. 


Table  B.  12.27 


Variable 

Group 

N 

Mean 

SD 

Sig.  Level 
(df=27) 

1 . Midterm 

Individual 

28 

4.4 

1.3 

.32 

Cooperative 

28 

4.1 

1.3 

2.  Final  Exam  - Midterm  Content  Only 

Individual 

28 

3.3 

0.9 

.55 

Cooperative 

28 

3.4 

0.9 

Research  Questions: 

1 . The  difference  between  means  for  the  individual  learning  environment  and  the  cooperative 
learning  environment  for  the  midterm  was  not  statistically  significant  beyond  the  .05  level  of 
significance. 

2.  The  difference  between  means  for  the  individual  learning  environment  and  the  cooperative 
learning  environment  for  the  final  exam  content  that  was  also  covered  on  the  midterm  was  not 
statistically  significant  beyond  the  .05  level  of  significance. 

This  lack  of  statistical  significance  indicates  that  a null  hypothesis  specifying  equal  effects  of  individual 
learning  and  cooperative  learning  in  this  context  is  still  a viable  hypothesis  and  cannot  be  rejected.  Some 
possible  reasons  for  this  result  could  be  related  to  classroom  context  and  operational  factors.  These  factors 
should  be  considered  and  discussed  by  the  experimenter. 
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Table  B.  12.28 


Paired  Samples  Statistics 


Mean 

N 

Std. 

Deviation 

Std.  Error 
Mean 

Pair  1 

Midterm  Individuai 

4.3929 

28 

1 .28638 

.24310 

Midterm  Cooperative 

4.1429 

28 

1 .32537 

.25047 

Pair  2 

Finai  MT  Content 
individuai 

3.2857 

28 

.93718 

.17711 

Finai  MT  Content 
Cooperative 

3.4286 

28 

.92009 

.17388 

Table  B.  12.29 


Paired  Samples  Correlations 


N 

Correlation 

Sig. 

Pair  1 

Midterm  Individuai  & 
Midterm  Cooperative 

28 

.509 

.006 

Pair  2 

Finai  MT  Content 
Individual  & Final  MT 
Content  Cooperative 

28 

.110 

.576 

Table  B.  12.30 


Paired  Samples  Test 


Paired  Differences 

t 

df 

Sig. (2- 
tailed) 

Mean 

Std. 

Deviation 

Std. 

Error 

Mean 

95%  Confidence  Interval 
of  the  Difference 

Lower 

Upper 

Pair  1 

Midterm  Individual  - 
Midterm  Cooperative 

.25000 

1 .29458 

.24465 

-.25198 

.75198 

1.022 

27 

.316 

Pair  2 

Final  MT  Content 
Individual  - Final  MT 
Content  Cooperative 

-.14286 

1.23871 

.23409 

-.62318 

.33746 

-.610 

27 

.547 
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RESEARCH  MANUSCRIPTS  REVIEW  AND  SUMMARY 
Jule  Dee  Scarborough,  Ph.D. 

(See  Portfolio  Sections  A.3  Research,  A. 5 Faculty  Development  Program,  and  B.11-12) 

Each  of  the  seven  professors  performed  classroom  research  on  the  Scholarship  of  Teaching. 
For  details  about  their  research  design  and  results,  see  B.  1 1,  a-g. 

The  Scholarship  of  Teaching,  or  experimental  classroom  research,  was  a variable  in  the 
research  and  evaluation  design  to  determine  the  success  of  the  CEET  Faculty  Development 
Program  (A. 5).  It  is  important  to  remember  that  our  primary  goal  was  to  prepare  the 
professors  for  experimental  classroom  research  on  teaching  and  learning.  That  type  of 
research  is  quite  different  from  research  typically  performed  by  engineers  and  technologists. 
The  research  was  based  upon  what  they  learned  in  the  Faculty  Development  Program  about 
teaching  and  student  learning,  including  the  program  component  on  educational  research. 
Based  upon  their  interests,  stemming  from  program  participation,  the  professors  worked  with 
the  program  leaders  to  determine  research  questions  and  the  experimental  study  designs. 

They  then  executed  the  research  in  their  respective  classrooms. 

Only  one  study  achieved  statistical  significance  in  student  learning;  the  others  did  not  achieve 
statistical  significance.  However,  our  primary  goals  were  to  engage  them  in  a program  during 
which  they  built  a knowledge  foundation  about  teaching,  student  learning  and  assessment  and 
to  prepare  them  to  engage  in  classroom  research  on  the  Scholarship  of  Teaching.  To  some,  the 
lack  of  statistical  significance  in  the  classroom  research  would  be  disappointing.  To  us,  the 
formal  execution  of  the  research  was  the  primary  and  most  important  aspect  of  the  research. 
As  mentioned  above,  the  design  and  execution  of  formal  experimental  research  in  the 
classrooms  with  students  as  the  Scholarship  of  Teaching  was  the  goal,  serving  as  a variable  in 
the  research  and  evaluation  design.  Also  research  studies  inform  us,  regardless  of  the 
statistical  results.  It  is  important  that  the  professors  learn  how  to  explain  not  only  what 
happens  when  statistical  significance  is  achieved  but  also  to  consider  and  understand  why 
statistical  significance  was  not  achieved.  This  is  what  leads  to  new  questions  or  hypotheses. 
Either  way,  the  research  and  its  results  will  inform  the  teaching  and  learning  communities  and 
contribute  to  those  bodies  of  knowledge  and  the  literature. 

We  were  pleased  with  the  professors’  efforts;  their  dedication  to  full  execution  of  formal, 
experimental  classroom  research;  and  their  follow  through  with  preparing  the  draft  research 
manuscripts  that  will  be  submitted  to  professional  journals  across  engineering  and  technology. 
All  seven  professors  who  participated  in  the  program  described  in  Portfolio  Section  A. 5 and 
research  described  in  Section  A.  3 submitted  draft  manuscripts.  A critique  of  the  manuscripts 
follows  the  Faculty  Eearning  Community’s  research  questions  below.  Please  note  that  two 
different  professors  researched  the  same  questions  presented  in  2 and  4 below.  Also  another 
two  different  professors  researched  the  same  questions,  presented  in  5 and  7 below.  The 
research  questions  across  all  Eearning  Community  members  were 

1 . Does  individual  performance-based  learning  or  cooperative  performance-based  learning  result  in  better 
student  learning  as  reflected  in  an  end-of-unit  exam. 
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2.  a.  Does  a performance  assessment  administered  in  conjunction  with  a traditional  test  result  in  increased 
learning  beyond  the  traditional  test  alone  as  indicated  by  scores  on  the  traditional  test? 

b.  Does  the  order  of  the  administration  of  a performance  assessment  and  a traditional  test  result  in 
differential  learning  as  indicated  by  the  combination  of  the  performance  assessment  and  traditional 
test  scores? 

c.  Does  the  order  of  administration  of  a performance  assessment  and  a traditional  test  affect 
knowledge  retention  as  indicated  by  a final  exam? 

3.  a.  In  problem-based  learning  projects,  are  hands-on  manipulative  procedures  more  effective  on  learning 
than  graphical  procedures  as  reflected  in  a midterm  exam? 

b.  In  problem-based  learning  projects,  are  hands-on  manipulative  procedures  more  effective  on 
retention  of  learning  than  graphical  procedures  as  reflected  in  a final  exam? 

4.  a.  Does  a performance  assessment  administered  in  conjunction  with  a traditional  test  result  in  increased 
learning  beyond  the  traditional  test  alone  as  indicated  by  scores  on  the  traditional  test? 

b.  Does  the  order  of  the  administration  of  a performance  assessment  and  a traditional  test  result  in 
differential  learning  as  indicated  by  the  combination  of  the  performance  assessment  and  traditional  test 
scores? 

c.  Does  the  order  of  administration  of  a performance  assessment  and  a traditional  test  affect  knowledge 
retention  as  indicated  by  a final  exam? 

5.  a.  Does  individual  learning  vs.  cooperative  learning  result  in  differential  knowledge  gains  as  indicated 
by  a traditional  test? 

b.  Does  individual  learning  vs.  cooperative  learning  result  in  differential  knowledge  retention  as 
indicated  by  a final  exam. 

6.  a.  Does  the  administration  of  a performance  assessment  that  covers  a subset  of  content  also  covered  in 
a traditional  midterm  result  in  increased  learning  beyond  the  administration  of  the  traditional  midterm 
alone  as  indicated  by  the  scores  on  the  traditional  midterm? 

b.  Does  the  administration  of  a performance  assessment  that  covers  a subset  of  content  also  covered  in 
a traditional  midterm  result  in  increased  knowledge  retention  beyond  the  administration  of  the 
traditional  midterm  alone  as  indicated  by  a separate  final  exam? 

7.  a.  Does  individual  learning  vs.  cooperative  learning  result  in  differential  knowledge  gains  as  indicated 
by  a traditional  test? 

b.  Does  individual  learning  vs.  cooperative  learning  result  in  differential  knowledge  retention  as 
indicated  by  a final  exam? 


2 


Discussion  of  Research  Manuscripts 

Variable  to  determine  success  of  the  CEET  E acuity  Development  Program 
As  the  final  evaluation  faetor  to  determine  or  eonfirm  what  professors  had  learned  in  the 
CEET  Eaeulty  Development  Program,  the  manuseripts  revealed  that  they  had  learned  what 
was  expeeted.  Although  some  of  the  drafts  did  not  reveal  the  literature  basis  or  learning  theory 
basis  for  the  researeh,  it  was  clear  in  the  execution  of  the  studies  that  they  understood  what 
they  were  trying  to  aehieve.  Also  some  of  the  professors  did  not  use  the  edueational 
terminology  at  all;  some  used  it  quite  well;  some  still  interehanged  terms  sueh  as  “objeetives” 
with  “outcomes.”  However,  once  again,  it  was  apparent  that  they  understood  what  they  were 
diseussing.  The  edueational  “jargon”  or  terminology  is  usually  confusing  to  those  who  have 
not  followed  its  transitions  and  often  subtle  definition  ehanges  with  those  transitions.  Overall, 
the  manuseripts,  along  with  the  other  eight  variables,  were  evidenee  of  program  sueeess. 

Quality 

The  quality  of  the  manuseripts  varied  from 

• excellent,  ready  to  submit  for  publication 

• excellent,  appropriate  draft;  almost  ready  to  submit 

• appropriate  for  draft;  eonsider  eritique  comments,  modify,  and  then  submit 

Challenges 

The  greatest  ehallenges  to  manuseript  preparation  for  these  professors  seem  to  be 

• use  of  educational  terminology  (for  all) 

• writing  style  (for  some) 

• establishing  the  context  of  the  study  within  the  framework  of  the  learning  theory;  other 
research,  and  relevant  literature  (for  all) 

• diseussing  and  interpreting  the  results  in  the  eontext  of  learning  theory,  other  researeh, 
and  relevant  literature  (for  all) 

• establishing  where  the  researeh  leads  them  next  (for  all) 

• explaining  and  soureing  the  context  of  their  study  within  the  Eaeulty  Development 
Program  (for  all,  however,  some  did  mention  it) 

• Closing  the  Loop:  none  of  the  professors  deseribed  ehanges  or  future 

Some  of  the  professors  prepared  their  manuseripts  using  a writing  style  more  appropriate  for 
teehnieal  or  lab  reports  - no  eontext.  Although  a few  did  somewhat,  most  did  not  use 
appropriate  edueational  terminology;  however,  it  is  important  to  remember  that  most  the 
edueational  terminology  was  very  new  to  almost  all  of  the  professors.  Some  professors  did  not 
provide  the  theoretieal  basis  for  their  research  using  other  researeh  and  literature  to  create  the 
researeh  eontext  and  framework;  others  introdueed  their  researeh  using  relevant  sourees, 
researeh  or  literature,  although  somewhat  minimally,  but  did  not  follow  through  with  using  it 
in  the  results,  discussion,  analysis,  or  conclusions  sections  of  their  manuscripts.  Most 
importantly,  none  of  the  professors  deseribed  where  the  research  would  lead  them;  they  did 
not  close  the  loop  and  deseribe  what  ehanges  they  would  make  in  their  eourses  or  what 
researeh  questions  would  be  eonsidered  for  the  future. 
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Comments  are  very  general  and  not  meant  to  serve  as  similar  to  the  Peer  Review  or  Refereed 
process  by  journal  boards.  Our  priority  was  on  establishing  the  overall  quality  of  research,  or 
should  the  professors  choose,  also  implementation  of  new  teaching  and  learning  strategies. 
Six  articles  focused  on  the  research;  one,  however,  focused  on  teaching  and  learning  - 
implementation.  Therefore,  it  was  not  possible  to  review  the  research  process  used  by  that 
professor,  only  the  student  learning  data  as  reported  in  Section  B.  1 1 . See  the  comments 
below  for  each  manuscript: 
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! = excellent  J = epprepriete  first  dreft 
NC  = needs  more  consideration 


Table  B.12.1:  Research  Manuscript  Review 


Professor 

Manuscript  Title 

Quality 

Research 

Questions 

Design 

Methods 

Procedures 

Literature 

Discussion  & 
Interpretation 

Conclusions  and  Recommendations 

Coller, 

Brianno 

Mechanical 

Engineering 

t 

• 

y 

An  experiment  in 
hands-on  learning 
in  engineering 
mechanics:  statics 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

1 . Consider  writing  a manuscript  this  well  describing  the 
program. 

2.  Use  more  education  and  learning  terminology 

3.  Close  the  Loop:  Also  where  are  you  going  from  here? 
What  changes  will  this  lead  you  to  make  in  the  course  next 
time?  Does  this  lead  you  to  reconsider  the  design  and 
research  again,  or  on  to  other  research  questions?  New 
hypotheses? 

Gupta, 

Abhijit 

Mechanical 

Engineering 

r 

• 

y 

Effect  of  order  of 
administration  of 
Performance 
Assessment  and 
Traditional 
Assessment 

Yes 

Yes 

Yes 

Yes 

Introduction 
good;  could 
choose  to 
expand, 
revealing 
more 
literature 
context  for 
your  study 

Needs  more 
discussion  and 
interpretation 
in  context  with 
current  or 
related 
research;  if 
none,  then 
establish  that 
there  seems  to 
be  no  other 
studies  similar 
to  this  one 

1.  Introduce  more  learning  theory;  theoretical  basis  for 
study 

2.  Introduce  program 

3.  Use  more  education  and  learning  terminology 

4.  Could  reference  literature  in  consideration  of  discussion 
about  results;  whether  that  means  adding  a “discussion  and 
interpretation”  section  before  results  and  conclusions??? 

5.  That  would  enhance  the  overall  effect  of  what  you  are 
presenting.  And  there  is  really  a lot  more  to  discuss. 

6.  Close  the  Loop:  Also  where  are  you  going  from  here? 
What  changes  will  this  lead  you  to  make  in  the  course  next 
time?  Does  this  lead  you  to  reconsider  the  design  and 
research  again,  or  on  to  other  research  questions?  New 
hypotheses? 

7.  edit  for  infinitives 

8.  use  listing  and  numbers  (e.g.,  research  questions,  indent 
and  number) 

9.  check  organization  and  structure 
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! = excellent  J = epprepriete  first  dreft 
NC  = needs  more  consideration 


Table  B.  12.  2:  Research  IV 

anu  script 

leview 

Professor 

Manuscript  Title 

Quality 

Research 

Questions 

Design 

Methods 

Procedures 

Literature 

Discussion  & 
Interpretation 

Conclusions  and  Recommendations 

Regina  D. 
Rahn  and 
Reinaldo  J. 
Moraga 

Industrial 

Engineering 

t 

• 

y 

The  Study  of 
Knowledge 
Retention  and 
Increased 
Learning  Through 
the  Use  of 
Performance 
Based  Tasks 

Yes 

Yes 

Yes 

Yes 

Introduction 
good;  could 
choose  to 
expand,  hut 
also  not 
necessary 

Needs  more 
discussion  and 
interpretation 
in  context  with 
current  or 
related 
research;  if 
none,  then 
establish  that 
there  seems  to 
be  no  other 
studies  similar 
to  this  one 

1.  Introduce  more  learning  theory;  theoretical  basis  for 
study 

2.  Use  more  education  and  learning  terminology 

3.  Explain  program 

4.  Could  reference  literature  in  consideration  of  discussion 
about  results;  whether  that  means  adding  a “discussion  and 
interpretation”  section  before  results  and  conclusions??? 

5.  That  would  enhance  the  overall  effect  of  what  you  are 
presenting.  And  there  is  really  a lot  more  to  discuss; 

6.  Close  the  Loop:  Also  where  are  you  going  from  here? 
What  changes  will  this  lead  you  to  make  in  the  course  next 
time?  Does  this  lead  you  to  reconsider  the  design  and 
research  again  or  on  to  other  research  questions?  New 
hypotheses? 

Reinaldo 
Moraga  and 
Regina 
Rahn 

Industrial 

Engineering 

T 

• 

y 

Studying 
Knowledge 
Retention  through 
Cooperative 
Learning  in  an 
Operations 
Research  Course 

Must  source 
literature  (e.g., 
into 

paragraph- 
coop  learning 
is  well  known 
in  literature); 
have  to  source 
these  type  of 
statements  and 
mention 
exactly  “what” 
about  coop 
learning; 

You  did  that 
later,  so  just 
source  up 
front; 

1.  Introduce  more  learning  theory;  theoretical  basis  for 
study 

2.  Use  more  education  and  learning  terminology 

3.  Explain  program 

4.  Could  reference  literature  in  consideration  of  discussion 
about  results;  whether  that  means  adding  a “discussion  and 
interpretation”  section  before  results  and  conclusions??? 

5.  That  would  enhance  the  overall  effect  of  what  you  are 
presenting.  And  there  is  really  a lot  more  to  discuss. 

6.  Close  the  Loop:  Also  where  are  you  going  from  here? 
What  changes  will  this  lead  you  to  make  in  the  course  next 
time?  Does  this  lead  you  to  reconsider  the  design  and 
research  again  or  on  to  other  research  questions?  New 
hypotheses? 
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NC  = needs  more  consideration 


Table  B.12.3:  Research  Manuscript  Review 


Professor 

Manuscript  Title 

Quality 

Research 

Questions 

Design 

Methods 

Procedures 

Literature 

Discussion  & 
Interpretation 

Conclusions  and  Recommendations 

R.  A. 
Tatara, 

PhD 

Technology 

y 

NC 

Applying  Detailed 
Student  Learning 
Objectives,  Group 
Learning,  and 
Assessments  in  an 
Introductory 
Polymers  Course 

Not 

mentioned 

in 

manuscript 

Yes 

Not  clear  in 
manuscript; 
number  or 
separate 
linearly; 

Introduce 
study;  set  it 
up  with  other 
research  and 
literature 
context; 

Identify  what 
you  are  going 
to  do  clearly; 

Present 

methodology 

and 

procedures; 

Not  clear  in 
manuscript; 
number, 
separate 

Some  used; 
but  some 
critical 
sources  not 
there;  need 
more, 
especially 
theoretical 
basis; 

So  much  more 
possible  in 
context  with 
literature; 

Good  stage 
setting  for 
course  change 
from  professor 
centered  to 
student 
centered; 

remember 
knowledge  and 
assessment 
centered 
complete  the 
balance; 

show  clear 
connections  to 
national 
standards 

IMPORTANT 
The  “reversed” 
and 

“intentional” 
process  is: 

a.  outcomes 

b.  assessments 

c.  teaching 
models,  etc.; 
see  page  3 
bottom 

1.  Background  introduces  the  college  and  program  context;  add 
Introduction  to  the  study 

2.  use  more  education  and  learning  terminology 

3.  need  literature  to  set  study  context 

4. **Introduce  more  learning  theory;  theoretical  basis  for  study; 
add  literature  to  interpret  study  results 

5.  break  away  somewhat  from  lab  report  type  writing  to  research 
manuscript  style 

6.  add  abstract  and  key  descriptors;  .consider  overall  organization 
and  structure 

7.  be  clear  about:  “student  learning  outcomes,”  NAIT  national 
standards  (whereas  ABET  has  labeled  theirs  “outcomes,”  we 
began  with  term  objectives  because  that  was  traditional  language 
and  the  way  some  had  it  in  their  course,  as  course  learning 
objectives,  etc.) 

8.  be  more  specific  about  change  process;  that  will  impress 
readers  and  help  them  to  understand  magnitude  of  change 

9.  must  source  (e.g.,  individual  vs.  group  learning 
accountability,  Johnson,  etc.) 

10.  source  program,  etc. 

11.  In  intro,  add  intro  to  CEET  program,  etc.;  that  is  the  context 
of  the  study 

12.  results  are  good,  but  what  does  this  mean  for  the  course  and 
other  courses?  Is  replication  needed?  Close  the  loop;  identify 
changes  for  next  time  teaching  course 

13.  change  fonts  and  size  in  graphics  or  boxes-reader  impact 

14.  Conclusions  must  be  addressed  related  to  theory  and  other 
studies  or  research;  connect  and  discuss,  then  interpret,  then 
conclude 

15.  Close  the  Loop:  Also  where  are  you  going  from 
here?  What  changes  will  this  lead  you  to  make  in  the 
course  next  time?  Does  this  lead  you  to  reconsider  the 
design  and  research  again  or  on  to  other  research 
questions?  New  hypotheses? 
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Table  B.12.4:  Research  Manuscript  Review 


Professor 

Manuscript 

Title 

Quality 

Research 

Design 

Methods 

Procedures 

Literature 

Discussion  & 

Conclusions  and  Recommendations 

Questions 

Interpretation 

Azad,  Abul 

t 

Suggest  new 

NA 

Ok  for  type 

NAfor 

NA  for 

You  have 

Expand  this 

1 . The  priority  of  the  manuscripts  was  to  prepare  a 

• 

titles  to  better 

of  article 

research. 

research. 

some  good 

section  and  use 

manuscript  about  the  classroom  research  as  that  is 

Engineering 

y 

reflect 

Implementation 

but  for 

but  for 

sourcing, 

literature  to 

the  primary  focus  of  the  initiative. 

Technology 

NC 

manuscript 

and  Program 

Appendixes 

presenting 

presenting 

but  can  use 

discuss  as 

content 

manuscript 

- good 

program 

program 

more 

similar. 

2.  Consider  using  more  sources  in  the  Intentional 

approach 

process, 

process,  ok 

literature 

dissimilar,  or 

Instructional  Design  section.  Use  Wiggins  for 

Appropriate 

for 

ok 

throughout 

how  we  have 

Reversed  Design,  Dick  and  Carey  for  Systematic 

Re-design  of  an 

approach: 

presenting 

gone  in  a new 

Design,  Jule  Scarborough  for  “Intentional”  or  all 

Introductory 

Digital 

Program 

products 

direction 

three  when  used  together. 

Electronics 

description  and 

3.  Edit  for  prepositions  and  infinitives 

within  an 

outcomes  was 

Electrical 

focus,  not 

4.  Note  that  Reflective  Practice  is  what  you  engaged 

Engineering 

classroom 

in  to  prepare  for  The  Scholarship  of  Teaching;  then. 

Technology 

research; 

continue,  as  you  do,  and  discuss  “Critical 

Undergraduate 

Reflection”  for  students;  Portfolio  Assessment  is  a 

Program 

good  example  for  Self-Assessment.  Use  some 
sources  and  check  understanding  and  meanings. 

Reflective 
Practice  - Part  I: 

5.  Must  source  appropriately  throughout;  e.g. 

Course  Re-design 

Assessment  System,  need  to  note  Kuhs  et  al  for 

in  Engineering 

basic  model,  then  program  and  Jule  for  extended 

and  Technology 

Resesarch  article 

model,  then  you 

Using 

will  be  prepared 

Educational 

as  second 

6.  Discuss  results;  itemize  changes,  then  express 

Theory, 

manuscript 

how  each  change  you  implemented  worked  or  did 

Research,  Best 

not  work.  Discuss  what  you  think  occutTed,  OR  why 

Practices:  Part  I 

it  seemed  to  work  or  why  not,  then. . . . 

The  Scholarship 

6.  Close  the  loop. . . .identify  changes  for  the  next 

of  Teaching 

time  based  upon  what  was  learned  during  this 

Through 

implementation.  How  did  the  students  respond  to  the 

Reflective 
Practice  - Part  II: 

new  course  strategies?  And,  more. 

Experimental 

7.  Later,  when  you  write  the  research  manuscript. 

Classroom 

also: 

Research  in 

. Close  the  Loop:  Also  where  are  you  going  from 

Engineering  and 

here?  What  changes  will  this  lead  you  to  make  in  the 

Technology 

course  next  time?  Does  this  lead  you  to  reconsider 
the  design  and  research  again  or  on  to  other  research 
questions?  New  hypotheses? 
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Table  B.12.5:  Research  Manuscript  Review 


Abdel- 

Motaleb 

Ibrahim 

Electrical 

Engineering 

y 

NC 

Instructional 
Research  Project: 
Learning  at  the 
Expert  Level 
“Investigation"  of 
the  Impact  of 
Cooperative  Based 
Learning  on 
Engineering 
Student  Learning  ” 

Yes 

Yes 

Yes 

Yes 

None;  really 
need  to  build 
study  context 
with  literature 

Also  need  to 
interpret  and 
discuss  using 
literature 

How  does 
your  study  fit 
into  the 
picture;  relate 
it  to  other 
studies;  or 
note  it  as  new 
or  different 

In  context 
of  results, 
but  not  in 
context  of 
related 
research  or 
theoretical 
literature 

1.  Need  introduction  to  study  and  program;  use  education  and  learning 
terminology 

2.  Use  more  education  and  learning  terminology 

3.  needs  literature  to  set  study  context 

**Introduce  more  learning  theory;  theoretical  basis  for  study 

4.  needs  literature  to  interpret  study  results 

5.  needs  to  introduce  study  with  questions,  design,  methods,  procedures 

6.  needs  to  break  away  somewhat  from  lab  report  type  writing  to 
research  manuscript  style 

7.  research  focus  needs  to  be  evident  in  title,  abstract,  or  body 

8.  be  clear  about: 

“student  learning  outcomes,”  NAIT  national  standards  (whereas  ABET 
has  labeled  theirs  “outcomes,”  we  began  with  term  objectives  because 
that  was  traditional  language  and  the  way  some  had  it  in  their  course,  as 
course  learning  objectives,  etc. 

9.  be  more  specific  about  change  process;  that  will  impress  readers  and 
help  them  to  understand  magnitude  of  change 

10.  must  source  (e.g.,  individual  vs.  group  learning  accountability, 
Johnson,  etc.) 

1 1 . need  to  source  program,  etc. 

12.  In  intro,  add  intro  to  CEET  program,  etc.;  that  is  the  context  of  the 
study 

13.  what  “uncontrolled  variables”  in  concluding  remarks;  this  would  be 
great  to  discuss 

14.  Use  boxes  or  graphics  to  present  rubric,  test  items,  etc.  to  separate 
them  from  other  language 

15.  design  graphics-lines  off 

16.  consider  overall  organization  and  structure 

17.  . Close  the  Loop:  Also  where  are  you  going  from  here? 
What  changes  will  this  lead  you  to  make  in  the  course  next 
time?  Does  this  lead  you  to  reconsider  the  design  and  research 
again  or  on  to  other  research  questions?  New  hypotheses? 
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Applying  Detailed  Student  Learning  Objectives,  Group  Learning, 
and  Assessments  in  an  Introductory  Polymers  Course 


R.  A.  Tatara,  PhD 
Department  of  Technology 
Northern  Illinois  University 
DeKalb,  Illinois  60115 


ABSTRACT 

“Materials  and  Processes  in  the  Plastics  Industry”  is  a three  credit-hour  course  at  Northern 
Illinois  University.  It  is  an  introductory  course  in  plastics  technology  designed  to  familiarize  the 
student  with  the  basic  polymers/plastics.  The  course  has  been  reorganized  to  focus  the  learning 
process  around  the  student  and  specific  student  learning  objectives  and  outcomes  connected  to 
curriculum  topics  and  various  types  of  learning  assessments.  Higher-level  objectives  that  stress 
designing,  planning,  judging,  and  analyzing  are  emphasized.  Teaching  is  partially  shifted  to 
group  learning  and  techniques  are  suggested  to  maximize  the  individual’s  performance  in  a 
group  setting.  Scoring  rubrics  are  part  of  the  assessment  system.  Traditional  examination 
questions  are  retained  but  now  each  question  is  connected  to  a specific  student  learning 
objective.  Specific  examples  of  learning  objectives,  syllabus  content,  a performance  task, 
assessment  rubric,  and  examination  questions  are  presented  for  this,  or  any,  introductory 
polymers  course.  Results  for  one  application  (semester)  of  this  process  are  also  presented. 


BACKGROUND 

At  Northern  Illinois  University  (NIU),  an  introductory  polymers  (plastics)  course  is 
offered  within  the  Department  of  Technology.  This  three  credit-hour  course  (TECH  344)  is  titled 
“Materials  and  Processes  in  the  Plastics  Industry”  and  is  a three  credit-hour  course.  The 
Department  of  Technology  is  part  of  the  College  of  Engineering  and  Engineering  Technology 
along  with  the  Departments  of  Electrical,  Industrial,  and  Mechanical  Engineering.  The 
Department  of  Technology  is  comprised  of  three  undergraduate  emphases:  Manufacturing 
Engineering  Technology,  Electrical  Engineering  Technology,  and  Industrial  Technology. 
Specifically  for  the  Industrial  Technology  students,  there  is  a plastics  area  of  study  and  TECH 


344  fulfills  one  of  the  required  classes.  Industrial  Technology  received  initial  accreditation  from 


the  National  Association  of  Industrial  Technologists  (NAIT)  in  1998  and  has  been  reaccredited 
during  the  2001/2002  academic  year;  the  program  is  now  accredited  through  2008.  TECH  344  is 
also  taken  by  undergraduate  students  from  Mechanical  Engineering,  Mechanical  Engineering 
Technology,  and  other  Industrial  Technology  study  areas  such  as  manufacturing  and  computer- 
aided  design. 


“Materials  and  Processes  in  the  Plastics  Industry"  is  an  introductory  course  in  plastics 
technology  designed  to  familiarize  the  student  with  the  basic  polymers/plastics  along  with  some 
plastics  fabrication  processes.  Topics  include 

History  of  Plastics; 

Basic  Concepts  in  Organic  Chemistry; 

Materials:  Thermoplastics,  Thermosets,  and  Elastomers; 

Properties; 

Additives,  Eillers,  and  Reinforcements; 

Eabricating  with  Plastics; 

Recycling,  Environmental  Aspects;  and 

Plastic  Processing  Methods. 

There  is  no  formal  laboratory  although  there  are  demonstrations,  in  the  laboratory,  of  some  of  the 
common  plastics  processing  methods.  (Plastics  processing  is  extensively  studied  in  another 
course,  TECH  345.)  Each  student  will  have  the  opportunity  to  learn  the  origin,  the  identity,  and 
the  characteristics  of  the  major  plastics  along  with  current  process  terminology  and  product 
applications . The  course  is  similar  to  introductory  polymer  science  or  polymeric  materials 
courses  elsewhere  in  chemical  engineering,  materials  science,  and  mechanical  engineering  or 
technology  departments  although  the  emphasis  in  TECH  344  is  more  on  the  properties  and 
application  of  polymers  rather  than  the  chemistry.  Eor  example,  the  basic  addition  and 
condensation  reactions  forming  polymers  are  included  in  the  curriculum  but  details  of  reaction 
rates  and  the  initiation,  propagation,  and  termination  reactions  are  not.  Typically,  students  meet 
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twice  a week  for  75  minute  sessions  during  a 16  week  semester  with  the  last  week  reserved  for 
final  examinations.  Historically,  students  in  TECH  344  are  exposed  to  an  instructor-centered 
command  style^^^  program  where  the  instructor  makes  all  the  decisions  and  determines  what  is 
taught  and  how  it  is  evaluated. 

In  2006,  in  an  effort  to  improve  student  instruction,  performance,  and  retention,  TECH 
344  has  been  reorganized.  Now  the  approach  to  learning  is  constructed  around  student  learning 
objectives  and  outcomes  connected  to  specific  curriculum  topics  and  various  type  of  learning 
assessments.  Various  teaching  and  learning  models  have  been  incorporated  with  higher- level 
objectives  of  Bloom’s  Taxonomy that  stress  designing,  planning,  judging,  and  analyzing  rather 
than  traditional  recalling,  classifying,  summarizing,  or  naming.  The  reorganization  of  TECH  344 
required  revising  the  course  content,  syllabus,  examinations,  lectures,  and  projects. 

STUDENT  LEARNING  OBJECTIVES 

The  first  step  in  the  reorganization  process  was  the  identification  of  each  pertinent 
student  learning  objective  (SEO)  and  ensure  that  course  topics  reinforced  the  objectives.  The 
curriculum  was  adjusted  to  reflect  all  these  changes.  In  the  next  step  various  teaching  and 
learning  models  were  incorporated  to  shift  the  program  from  instructor-centered  to  more  student- 
centered  where  individually,  or  in  groups,  students  take  responsibility  for  the  learning  process. 
Einally,  student  assessments  were  developed  to  judge  how  well  objectives  are  attained.  This 
required  analysis  of  previous  course  examinations,  revision  of  these  examinations,  and 
development  of  new  performance  tasks  that  provide  alternative  assessments  of  the  same 
knowledge.  Rubrics  were  constructed  to  evaluate  the  performance  tasks. 
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Two  sets  of  learning  objectives  are  present:  general  engineering  NAIT  and  course- 
specific  ones.  For  instance,  a NAIT  learning  outcome  or  objective  is: 

Apply  current  knowledge  and  adapt  to  emerging  application  of  math,  science,  engineering, 
and  technology. 

To  achieve  this  outcome,  TECH  344  includes  the  SLOs  and  sub-objectives  of: 

A.  Students  will  Describe  the  Fundamental  Structure  of  Plastics: 

A.l.  Students  will  interpret  and  draw  polymer  chains. 

A.l.a.  Students  will  compare  polymerization  reactions. 

A.l.b.  Students  will  compare  and  contrast  functional  groups  and  tacticity. 

A.l.c.  Students  will  describe  chain  topology. 

A. 2.  Students  will  compare  and  contrast  thermoplastics  and  thermosets. 

A. 2. a.  Students  will  select  commodity  and  engineered  plastics. 

A.2.b.  Students  will  differentiate  crystalline  and  amorphous  plastics. 

A.  3.  Students  will  name,  draw,  and  identify  elastomers. 

A. 3. a.  Students  will  explain  elastomers. 

A.3.b.  Students  will  summarize  polyisoprene. 

A. 3.C.  Students  will  select  and  qualify  other  elastomers. 

B.  Students  will  Predict  Plastics  Properties: 

B. l.  Students  will  describe  effects  of  structural  features  on  plastics  properties. 

B. l.a.  Students  will  quantify  and  solve  molecular  weight  distribution. 

B.l.b.  Students  will  qualitatively  evaluate  crystallinity  effects. 

B.2.  Students  will  distinguish  and  explain  mechanical,  physical,  thermal,  environmental, 
electrical,  and  optical  properties. 

B.2.a.  Students  will  select  ASTM  techniques. 

B. 3.  Students  will  explain  interactions  of  modifiers. 

B. 3.a.  Students  will  classify  additives,  fillers,  and  reinforcements. 

C.  Students  will  Describe  Plastics  Design  and  Finishing  Processing: 

C. l.  Students  will  differentiate  design  methods. 

C.2.  Students  will  classify  ways  of  assembling  plastics. 

C. 2.a.  Students  will  select  machining  methods. 

C.3.  Students  will  explain  methods  of  finishing  plastics. 

C. 3.a.  Students  will  give  examples  of  joining  and  decorating. 

C. 4.  Students  will  compare  and  contrast  rapid  prototyping  procedures. 

D.  Students  will  Recognize  the  Environmental  Aspects  of  Plastics: 

D. l.  Students  will  explain  waste  reduction  techniques. 

D. l.a.  Students  will  evaluate  source  control,  recycling,  regeneration,  degradation, 
landfills,  and  incineration. 

E.  Students  will  Analyze,  in  Depth,  Specific  Plastics  Topic: 
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E.l.  Students  will  construct  the  history  of  a plastics  topic,  or 
E.2.  Students  will  differentiate  a plastic,  or 
E.3.  Students  will  detail  a plastics  processing  method,  or 
E.4.  Students  will  describe,  in  detail,  a plastic  product. 

Appropriate  textbook  and  lecture  materials  were  selected  to  meet  these  objectives  and  the 


syllabus  recorded  the  order  and  progress  of  meeting  objectives  and  outcomes. 


GROUP  LEARNING  ORIENTED  SYLLABUS 

A syllabus  was  planned  that  placed  some  of  the  burden  of  learning  on  the  students.  It 
combined  instructor-centered  instruction  followed  by  student-centered  group  learning.  As  an 
example,  part  of  student  learning  outcome  A.  is: 


A.  2.  Students  will  compare  & contrast  thermoplastics  & thermosets, 
a.  Students  will  select  commodity  and  engineered  plastics. 

while  the  corresponding  syllabus  entries  are: 


Week  and  Objectives 

1 

Day  1 Topics,  Activities,  and  Due  Dates 

1 

Day  2 Topics,  Activities,  and  Due  Dates 

Week  #6 

Select  Thermosets 

Differentiate  a Plastic 

Commodity  Thermosets. 

Engineered  Thermosets. 

Read  Chapter  9 of  Textbook  due  10/9. 

Group;  Students  Differentiate  Engineered 
Thermosets  due  10/9. 

Homework;  Chapter  8 Questions  (evens)  due 
10/11. 

Week  #7 

Name,  Draw,  & Label 
Elastomers 

Polyisoprene. 

Group;  Students  Differentiate  Polyisoprene 
due  10/11. 

Other  Elastomers. 

Performance  Task  2 due  1 1/6. 

Homework;  Chapter  9 Questions  (evens)  due 
10/18. 

These  two  weeks  represent  four  class  periods.  In  the  first,  day  I of  week  6,  the  focus  is  on 
instruction  by  lecture  with  the  topic  being  commodity  (common)  thermosetting  resins.  But  day  2 
of  week  6 represents  a shift  as  students  in  groups,  following  the  format  examples  from  the 
instructor’s  lectures,  construct  their  own  set  of  lecture  notes  for  engineered  thermosetting  resins. 
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A due  date  for  this  assignment  is  given  as  10/9  along  with  an  individual  homework  assignment  to 
further  reinforce  the  material  by  presenting  the  same  information  from  a different  source  to  take 
advantage  of  various  learning  styles  among  students.  (A  similar  approach  is  used  in  prior  weeks 
for  thermoplastics.) 

In  this  regards,  it  is  vital  that  any  groups  are  legitimate  cooperative  learning  groups  where 
students  are  randomly  assigned  and  outperform  reasonable  expectations  by  their  combined 
efforts.  Additionally,  each  individual  in  the  group  must  be  independently  evaluated.  Examples  to 
accomplish  this  include  keeping  the  group  size  small,  giving  written  or  oral  examinations  to 
students,  and  observing  students  as  they  interact  within  their  group.  There  are  systematic 
techniques  available  to  maximize  the  individual’s  performance  in  a group  setting^^’"^l  These 
techniques  cover  various  ways  of  forming  groups,  including  ensuring  that  the  groups  are  random 
and/or  balanced.  Different  ways  of  group  functioning  and  dynamic  interaction  are  also 
documented.  A sampling  includes  rounds  where  students  take  turns  speaking;  group 
investigation  where  each  group  is  free  to  choose  a subtopic  within  the  area  of  study;  discussions 
where  students  take  opposing  sides  of  an  issue;  and  brainstorming  to  encourage  free  thinking  and 
rapid  development  of  ideas. 

In  any  case,  the  vital  elements  of  group  learning  are  to  assign  personal  responsibility  to 
each  student  along  with  individual  accountability.  A group  must  actually  engage  in  learning,  not 
just  doing  a task  or  assignment;  this  requires  the  group  to  produce  a product  at  the  end  of  the 
session  and  the  product  must  be  assessed  against  very  specific  criteria.  In  the  end,  each  student 
will  perform  at  a level  above  their  individual  capability,  benefiting  from  the  group  learning 
process. 
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Week  7 continues  the  process  for  elastomeric  thermosets.  Furthermore,  week  7 contains 
an  additional  assignment  labeled  Performance  Task  2 which  provides  another  way  for  students  to 
fulfill  their  learning  objective  of  being  able  to  understand  specific  polymers  and  addresses 
student  learning  objective  E:  Students  will  Analyze,  in  Depth,  Specific  Plastics  Topic. 
Performance  Task  2 is  also  described  in  the  TECH  344  syllabus: 

You  are  charged  with  critiquing  a specific  plastic  (or  polymer)  in  detail.  The  focus  is  on  what 
differentiates  the  plastic  from  other  plastics  by  describing  the  features  that  make  the  plastic 
unique.  The  features  would  involve  molecular  structure,  properties,  and  industrial,  commercial, 
or  consumer  uses  and  applications.  You  must  collect  the  appropriate  information,  coordinate 
your  findings,  judge  the  data,  and  write  a report. 

You  will  research  the  plastic  from  a variety  of  sources  - internet,  magazines,  journals, 
manufacturers  ’ datasheets,  product  literature,  books,  and  conference  proceedings.  Keep  track  of 
all  your  research  sources  and  be  prepared  to  report  on  which  were  more  useful.  The  research 
should  include  history  of  the  plastic/polymer;  fundamental  molecular  structure;  molecular  chain 
topology;  molecular  structural  features  and  effects  on  properties;  molecular  weight 
distributions;  basic,  general  mechanical,  physical,  and  chemical  properties;  and  uses  and 
applications. 

Devise  a procedure  for  collecting  your  information  prior  to  beginning  the  project. 

Choose  which  informational  sources  you  will  consult.  Write  down  the  procedure  in  a step-by- 
step  order.  Make  sure  proper  documentation  is  maintained  by  complete  referencing;  use  a 
system  of  referencing  from  a writing-style  guide/handbook. 

Present  your  findings  in  a written  research  report.  Include  data,  tables,  figures, 
diagrams,  charts,  graphs,  references,  and  photos,  where  appropriate,  to  better  illustrate  your 
findings.  The  report  must  have  a minimum  of  four  pages  of  text  (exclusive  of  references  and 
illustrations:  tables,  graphs,  figures,  charts,  photos,  etc.)  and  be  double-spaced  with  a 12-point 
font.  The  report  should  document  and  explain  the  plastic  or  polymer  and  its  unique  features  and 
uses  that,  in  your  judgment,  differentiate  it  from  other  plastics  and  polymers. 

With  lecture  information,  group  learning,  and  homework  assignments  all  reinforcing  each  other 

in  the  learning  process,  the  student  is  ready  to  proceed  to  upper  levels  of  Bloom’s  taxonomy;  this 

performance  task  requires  a higher-level  of  thought  requiring  judgment,  evaluation  and  planning. 

Notice  how  the  task  is  student-centered  and  uses  verbs  that  promote  action  by  the  student. 


ASSESSMENT 
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Of  course  to  objectively  assess  student  ability,  it  is  necessary  to  provide,  ahead  of  time,  a 
scoring  rubric  specific  for  this  assignment.  The  rubric  covers  research,  writing,  organization,  and 
quality  criteria  of  the  work.  A partial  rubric  for  Performance  Task  2 is: 


Criterion 

Reference 

Use 

Score  = 3;  Good 

Pertinent  references  are  used; 
properly  listed  at  end  of  report; 
and  clearly  cited  within  body  of 
report. 

Score  = 2:  Accentable 

Mostly  pertinent  references 
are  used  and  properly  listed  at 
end  but  much  text  is  not  cited. 

Score  = 1:  Weak 

Little  use  of 
references. 

Spelling/ 

Grammar 

Few  errors. 

Some  errors. 

Many  errors 
every  page. 

Organization 

Topics  are  logically  arranged 
with  good  flow  between  them. 
It  is  easy  to  follow  lines  of 
reasoning. 

Sometimes  difficult  to  follow  topics 
and  lines  of  reasoning. 

Topics  are  mostly 
disorganized; 
hard  to  follow 
reasoning. 

Critical 
findings 
or  data. 

Interprets  results  and  data  and 
properly  applies  interpretations 
to  the  conclusions. 

Interprets  results  and  data  but 
does  not  successfully  apply 
interpretations  to  conclusions. 

No  interpretation 
of  results  or  data. 
No  application  to 
the  conclusions. 

Among  assessment  tools  are  examinations  but  these  must  truly  test  the  material  presented  in 
class  and  be  consistent  with  the  SLOs.  To  accomplish  this,  an  extensive  test  bank  of  questions 
was  formulated.  Question  format  was  varied  with  the  emphasis  on  questions  that  are  objectively 
evaluated.  Thus  the  format  uses  multiple  choice,  fill-in-the-blank,  item  matching,  short  answer, 
and  true/false;  it  is  important  that  lower-order  as  well  as  higher-order  thinking  questions  are 
included  Each  test  bank  item  may  be  characterized  as  knowledge,  comprehension, 
application,  analysis,  evaluation,  or  synthesis,  consistent  with  Bloom’s  taxonomy.  Here  analysis, 
evaluation,  and  synthesis  represent  high-order  thinking  test  items  and  some  test  questions  ought 
to  be  at  such  levels  although  higher-order  problems  solving  may  be  better  assessed  through 
performance  tasks  and  their  rubrics.  For  TECH  344,  an  excerpt  from  the  test  bank  gives  some 
questions  related  to  basic  organic  chemistry.  The  nine  sample  questions  are  grouped  under 
student  learning  objective  A. i.-  Students  will  draw  and  explain  basic  organic  molecules. 

1 . The  type  of  bond  between  two  carbon  atoms  of  a polymer  is 


a)  covalent. 

b)  ionic. 

c)  dipolar. 

d)  metallic. 

**  For  the  next  6 items,  write  the  typical  number  of  bonds  the  element  makes  in  the  blank  space 
next  to  the  element. 

2.  hydrogen  

3.  sulfur  

4.  chlorine  

5.  carbon  

6.  fluorine  

7.  nitrogen  

8.  The  molecular  weight  of  heptane  is  a)  greater  than  the  molecular  weight  of  heptene. 

b)  the  same  as 

c)  less  than 

9.  Name  the  straight,  four  carbon-long  chain  molecule  containing  only  hydrogen  and  single 

bonds: . 


By  grouping  test  questions  under  SLOs,  direct  evidence  is  given  that  a specific  area  of 
knowledge  is  tested  for.  This  is  a useful  asset  in  national  educational  accreditation  programs  as  it 
provides  proof  of  learning,  once  test  scores  and  data  become  available.  Quiz  as  well  as  midterm 
and  final  examination  questions  are  selected  from  the  test  bank. 


RESULTS 

Results  for  one  application  (semester)  of  this  process  are  presented.  The  results  are 
grouped  by  significant  student  products  such  as  exams  and  performance  tasks. 

Midterm  Examination: 

To  improve  on  midterm  scoring  and  in  addition  to  lectures  and  homework  assignments, 
the  class  was  divided  into  two  groups  to  assess  the  difference  between  cooperative  versus 
individual  learning  in  four  content  areas.  The  content  areas  were  commodity  thermoplastics. 
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engineered  thermoplastics,  thermosets,  and  elastomers.  Random  assignment  of  students  to  two 
groups  allowed  us  to  assume  the  groups  were  equivalent.  Each  group  had  approximately  15 
students  while  each  small  learning  group  was  composed  of  3 students.  The  actual  delivery  of  the 
treatment  conditions  alternated  across  content  areas  and  groups.  An  outline  of  the  experimental 
model  is  provided: 


Individual  Learning  vs.  Cooperative  Learning 


Treatment 

Posttest  1 

Posttest  2 

Individual 
Learning  Group 

Individual  Learning 

Midterm  10/18/06 

Final  12/1 1/06 

Cooperative 
Learning  Group 

Cooperative 

Learning 

Midterm  10/18/06 

Final  12/1 1/06 

Treatment  " 

Group 

Content  Area 
I - Commodity 
Thermoplastic 
Study 
Summary 
Questions 

Content  Area 
II  - Engineered 
Thermoplastic 
Study 
Summary 
Questions 

Content  Area 
III  - Thermoset 
Study 
Summary 
Questions 

Content  Area 
IV  - Elastomer 
Study 
Summary 
Questions 

1 

Individual  - 15 

Cooperative 
Groups  #l-#5 

Individual  -15 

Cooperative 
Groups  #l-#5 

2 

Cooperative 
Groups  #l-#5 

Individual  - 15 

Cooperative 
Groups  #l-#5 

Individual  - 15 

From  an  item  analysis  viewpoint,  I did  not  eliminate  any  items  on  the  midterm;  it  only 
had  two  questions  (out  of  30)  considered  for  elimination.  Both  of  these  had  low  Item  Difficulty 
(11%  and  21%)  and  Low  Item  Discrimination  (-0.34  and  0.20).  A closer  examination  of  the  two 
questions  revealed  that  they  were  based  on  reading  assignments  and  items  not  covered  by 
lectures,  student  group  assignments,  or  performance  tasks.  But  the  items  were  judged  to  be 
reasonable  thus  rather  than  eliminating  them,  I emphasized  that  the  final  would  also  include 
questions  on  the  reading  of  chapters.  (This  includes  material  not  covered  in  lectures.)  It  was  also 
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noted  that  students  did  not  perform  better  on  the  items  from  the  four  content  areas  related  to  the 
cooperative  versus  individual,  traditional  cognitive  learning  activities.  Thus  this  led  to  another 
change  to  include  more  retention  questions  in  the  final  than  originally  planned.  A total  of  eight 
retention  questions  were  included  in  the  final,  two  from  each  of  the  four  content  areas. 

In  future  courses,  it  would  be  beneficial  to  emphasize  that  students  are  responsible  for 
chapter  readings  as  well  as  lecture  materials.  Also  other  group  learning  models  should  be  tried. 
There  are  systematic  techniques  available  to  maximize  the  individual’s  performance  in  a group 
setting.  These  techniques  cover  various  ways  of  forming  groups,  including  ensuring  that  the 
groups  are  random  and/or  balanced.  Different  ways  of  group  functioning  and  dynamic 
interaction  are  also  documented.  A sampling  includes  rounds  where  students  take  turns  speaking; 
group  investigation  where  each  group  is  free  to  choose  a subtopic  within  the  content  area  of 
study;  discussions  where  students  take  opposing  sides  of  an  issue;  and  brainstorming  to 
encourage  free-thinking  and  rapid  development  of  ideas.  Overall,  individuals  should  benefit  from 
the  group  learning  process  but  this  did  not  occur  presently. 

Final  Examination; 

Due  to  rewriting  of  items  based  on  experiences  from  the  midterm,  the  overall  class 
performance  on  the  final  examination  was  10%  better  than  the  midterm.  There  were  three 
questions  (out  of  50)  with  low  Item  Difficulty  (14%  and  two  at  21%).  However  two  of  these  had 
reasonable  Item  Discriminations  (0.44  and  0.47)  while  the  third  was  at  -0.34.  This  third  item  was 
discussed  in  lecture  and  included  in  the  textbook  readings  so  that  there  is  no  good  explanation  of 
why  the  item  proved  to  be  so  difficult;  only  this  item  was  considered  for  elimination.  Three  items 
had  Item  Difficulty  scores  of  100%;  these  items  show  no  discrimination  and  also  could  be 
considered  for  dropping.  But,  in  the  end,  all  items  were  retained  as  a database  to  be  expanded  as 
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future  exams  give  more  guidance  as  when  to  eliminate  items.  Certainly,  it  is  expected  that  the 
experiences  from  these  two  examinations  will  provide  for  better  future  test  items. 

There  was  even  better  improvement  on  the  eight  retention  questions;  the  students  scored 
12%  better  on  the  retention  questions  than  on  the  final  as  a whole.  This  indicates  that  if  special, 
or  extra,  attention  is  given  to  critical  topics,  students  are  able  to  perform.  Different  teaching 
and/or  learning  models  ought  to  be  considered  for  such  topics.  This,  in  conjunction  with  better 
group  learning  processes,  should  increase  test  performance,  including  performance  on  retention 
items. 

Performance  Assessments: 

Three  performance  assessment  tasks  were  assigned.  Generally,  scores  were  better  on 
these  than  the  standard  tests  as  the  tasks  gave  students  an  alternative  opportunity  to  give 
evidence  of  their  knowledge.  The  tasks  also  tested  the  students  at  higher  levels  of  Bloom’s 
taxonomy.  One  of  the  reasons  for  the  success  of  these  was  the  presence  of  rubrics  that  were  a 
great  asset  in  the  execution  of  the  performance  assessment  tasks.  Students  clearly  knew  the 
expectations  and  tailored  their  work  to  fulfill  the  requirements.  This  led  to  higher  scoring  and  the 
better  scores  were  justified. 

The  third  task  included  group  activities.  It  appears  that  students  working  in  groups 
towards  performance  tasks  benefit  more  than  groups  preparing  for  the  midterm  and  final 
examinations.  Students  seem  to  do  better  in  group  settings  where  more  creative,  open-ended 
projects  are  the  goal.  Future  courses  should  explore  and  exploit  this  trend.  Or  a cooperative 
versus  individual  learning  study  for  performance  tasks  could  be  conducted. 
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CONCLUDING  REMARKS 


After  several  offerings  of  TECH  344  with  its  revised  format,  more  data  should  be 
collected  to  determine  whether  or  not  student  learning  has  improved.  Test  scores  and  grades 
from  this  initial  offering  is  a baseline.  However,  if  there  remain  too  many  uncontrolled  variables, 
meaningful  quantitative,  statistical  findings  will  be  difficult.  Nevertheless,  even  qualitative  data 
and  instructor  experience  should  provide  an  evaluation  of  the  program.  Of  course  reorganization 
of  TECH  344  is  expected  to  be  a continual  process  as  more  feedback  and  information  arrives 
each  time  the  course  is  given. 
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1.  Introduction 

One  of  the  major  issues  of  educational  research  is  to  enhance  students’  learning.  As  identified 
(AACU,  2002;  Fox  and  Hackerman,  2003;  Bransford  et  ah,  1999),  the  factors  that  enhance 
students’  learning  are:  a)  enable  students  to  become  empowered,  informed,  and  responsive 
learners  ready  to  assume  a productive  role  in  society;  b)  improve  assessment  of  learning 
outcomes,  teach  a broader  range  and  larger  number  of  students,  provide  engaging  laboratory  and 
field  experiences,  and  enhance  faculty  knowledge  of  research  on  effective  teaching;  c)  work  with 
students’  pre-existing  knowledge,  teach  subject  matter  in  depth  and  provide  examples,  help 
students  to  develop  self-monitoring  and  reflection  skills,  and  integrate  these  practices  into  the 
curriculum  in  a variety  of  subjects;  and  d)  share  ideas  and  materials  so  that  projects  build  on, 
connect  to,  and  enhance  the  work  of  others. 

To  address  these  issues,  scholarship  of  teaching  has  receiving  attention  and  there  has 
been  much  debate  about  what  sort  of  teaching  encourages  effective  learning  (Biggs,  1996,  1999; 
Ramsden,  1992;  and  Prosser  and  Trigwell,  1999).  In  a simple  statement,  scholarship  in  teaching 
is  an  effort  to  make  the  teaching  process  transparent  so  that  one  can  understand  how  learning 
takes  place  (Trigwell,  2007).  For  this  to  happen,  teachers  must  be  informed  of  the  theoretical 
perspectives  and  literature  of  teaching  and  learning  in  their  discipline,  and  be  able  to  collect  and 
present  rigorous  evidence  of  their  effectiveness.  One  of  the  ways  to  achieve  these  is  to  adopt 
intentional  instructional  design  with  reflective  practice. 

1.1  Intentional  Instructional  Design 

Intentional  instructional  design  is  the  systematic  development  of  instructional  specifications 
using  learning  and  instructional  theory  to  ensure  the  quality  of  instruction.  It  is  the  entire  process 
of  analysis  of  learning  needs  and  goals  and  the  development  of  a delivery  system  to  meet  those 
needs.  Instructional  design  is  the  science  of  creating  detailed  specifications  for  the  development, 
preparation  of  instructional  materials,  implementation,  evaluation,  and  maintenance  of  situations 
that  facilitate  the  learning  of  both  large  and  small  units  of  subject  matter  at  all  levels  of 
complexity. 

Instructional-design  theory  includes  instructional  outcomes,  conditions,  methods,  and 
values.  Instructional  outcomes  include  both  results  that  are  intentional  and  those  that  are 
incidental.  This  includes  the  instruction's  effectiveness,  efficiency,  and  appeal  and  should  not  be 
confused  with  learning  outcomes.  Instructional  outcomes  focus  on  the  degree  of  success  in 
attaining  the  desired  learning  outcomes  (the  effectiveness  of  instruction)  but  also  include  the 
efficiency  and  appeal  of  the  instruction.  Instructional  conditions  are  factors  beyond  the 
influence  of  the  instructional  designer  that  impact  upon  the  effects  of  the  methods  of  instruction. 
Conditions  may  include  the  nature  of  what  is  being  learned  (the  content),  the  learner,  the  learning 
environment,  and  the  instructional  development  constraints  (resources).  Instructional  methods 
are  the  "how  to"  facilitate  human  learning.  They  are  the  elements  of  guidelines  that  inform 
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designers  and  teaehers  what  to  do  to  help  students  learn.  Instruetional  methods  are  situational 
rather  than  universal.  This  means  that  there  are  values,  desired  instructional  outcomes,  and 
instructional  conditions  (collectively  referred  to  as  instructional  situations)  in  any  context  that 
influence  whether  or  not  a given  instructional  method  should  be  used.  Also,  instructional 
methods  are  probabilistic  rather  than  deterministic.  That  is,  their  use  can  only  increase  the 
probability  that  the  desired  outcomes  will  be  attained.  Instructional  values  are  an  individual's  or 
group's  philosophy  or  beliefs  about  instruction. 

1.2  Reflective  Practice 

The  reflective  practice  is  one  of  the  major  components  of  scholarship  in  teaching.  There  are 
three  activities  that  support  the  facilitation  of  reflective  practice:  self  and  peer  assessment, 
problem-based  learning  and  personal  development  planning.  One  needs  to  decide  which 
approach  will  best  meet  for  a specific  need  given  the  course  objectives  and  context. 

Self  and  Peer  Assessment:  Both  self  and  peer  assessment  can  be  used  to  support  reflective 
practice,  since  they  involve  students  thinking  about  their  own  learning.  According  to  the 
pioneering  work  of  Bond,  self  and  peer  assessment  is  the  "involvement  of  students  in  identifying 
standards  and/or  criteria  to  apply  to  their  work  and  making  judgments  about  the  extent  to  which 
they  have  met  these  criteria  and  standards"  (Bond,  1995).  It  implies  that  self  assessment 
involves  two  clear  stages:  a)  the  identification  (and  learner  understanding)  of  standards  and 
criteria  for  a given  task  and  b)  the  making  of  one's  own  judgments  against  those  criteria. 

Self  assessment  can  be  used  to  facilitate  both  a process  of  learning  and  an  assessment 
product.  It  can  be  used  in  an  informal  way  to  encourage  students  to  think  about  their  work  and 
what  they  know  in  a given  subject.  The  evidence  of  development  through  self  assessment  can  be 
expressed  through  writing  a learning  diary  or  portfolio. 

As  the  name  suggests,  peer  assessment  involves  students  making  judgments  about  the 
quality  of  each  other's  work  in  relation  to  an  agreed  criteria.  This  is  a useful  tool  for  supporting 
reflective  practice,  as  it  focuses  on  dialogue  and  shared  interpretations  of  teaching  and  learning 
between  teachers  and  students  (Stefani,  1998).  Students  learn  from  each  other  and  use  the 
feedback  provided  by  peers  to  inform  their  own  learning.  Peer  assessment  enables  students  to 
understand  and  communicate  ideas  that  they  consider  important  with  the  lecturer  and  their  peers 
about  what  should  be  assessed  and  what  weighting  should  be  given  to  each  specific  criterion 
(Tribe  and  Tribe,  1986;  Hinett  and  Thomas,  1999).  Using  this  approach  student  is  encouraged  to 
make  qualitative  comments  about  the  work  of  their  peers.  To  promote  a sense  of  ownership  over 
the  process,  students  may  also  be  asked  to  attribute  a grade  to  the  work  and  experience  suggests 
that  students  prefer  and  enjoy  in  assigning  a grade  as  well  as  providing  comments  (Bond  and 
Tyree,  1980).  Research  findings  suggest  that  in  cases  where  they  are  involved  in  grading  there  is 
often  more  congruence  between  the  student  and  tutor’s  mark  (Stefani,  1994;  Bond  and 
Palchikov,  1989). 

The  key  to  using  self  and  peer  assessment  is  to  ensure  that  each  student  is  given  the  same 
opportunities  to  discover  how  they  learn.  At  the  same  time  it  is  a good  idea  not  to  streamline  the 
process,  but  allow  students’  need  to  discover  for  themselves  what  they  know  and  don't  know  and 
to  make  their  own  connections  if  these  processes  are  to  support  reflection. 

Problem-Based  Learning:  Problem-based  learning  (PBL)  is  used  as  a way  of  engaging  students 
in  real  problems.  Unlike  conventional  teaching,  PBL  starts  with  a problem  and  requires  the 
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student  to  research,  select,  analyze,  and  apply  information  and  theories  to  solve  it.  Students 
work  in  groups  or  teams  to  solve  or  manage  these  situations,  but  they  are  not  expected  to  acquire 
a predetermined  series  of  right  answers.  Instead  they  are  expected  to  engage  with  the  complex 
situation  presented  to  them  and  decide  what  information  they  need  to  learn  and  what  skills  they 
need  to  gain  in  order  to  manage  the  situation  effectively  (Savin-Baden,  2000). 

The  advantage  of  students  working  upon  real  or  simulated  situations  is  that  real  problems 
do  not  have  simple  solutions,  but  require  comparison  and  analysis  of  resources,  strategies  and 
costs.  As  such  the  learner  has  to  develop  skills  of  retrieval,  selection  and  discrimination  in  order 
to  solve  the  problem.  Students  also  develop  group  working  skills  as  they  work  together  to  solve 
a common  problem.  PBL  facilitate  a dialogue  between  the  student,  tutor,  and  peers  (and  in  some 
cases  external  parties),  which  helps  the  individual  make  sense  of  his  or  her  learning.  Laurillard, 
outlines  that  dialogue  has  three  important  functions  for  learning;  firstly  it  reveals  the  students' 
and  teachers'  conceptions  to  each  other,  secondly  it  provides  space  for  negotiation  and  adaptation 
of  topic  and  task  goals,  and  thirdly  and  perhaps  most  importantly,  it  provides  opportunity  for 
feedback,  reflection,  and  action  upon  feedback  (Laurillard,  1993). 

PBL  offers  a genuine  experience  or  context  in  which  reflection  can  take  place,  and  unlike 
traditional  problem  solving  where  the  student  is  directed  towards  appropriate  resources,  PBL 
forces  students  to  think  on  their  feet  and  draw  on  previous  experience  to  transfer  to  new  settings 
(Bond  and  Feletti,  1997). 

Personal  development  planning:  Personal  development  planning  (PDP)  aims  to  provide  students 
with  a structure  for  thinking  about  and  planning  their  own  development  and  can  be  seen  as  a 
process  of  evidencing  learning  and  reflection.  Portfolios  and  records  of  achievement  are  the 
common  forms  in  which  the  PDP  process  is  presented.  The  advantage  of  PDP  is  that  it  provides 
a rounded  picture  of  the  capabilities  of  an  individual.  Usually  consisting  of  three  parts  (a 
checklist  of  skills  or  competences  achieved,  evidence  of  achievement,  and  a reflective  piece  on 
how  the  skill  has  been  developed),  PDP  offers  more  information  than  a certificate  and  engages 
students  in  a process  of  thinking  about  their  learning.  Portfolios  can  be  used  both  for 
certification  purposes  and  as  an  additional  form  of  evidence  to  employers  and  educational 
institutions.  It  is  intended  that  PDP  will  help  students  to:  become  more  effective,  independent 
and  confident  self-directed  learners,  understand  how  they  are  learning  and  relate  their  learning  to 
a wider  context,  improve  their  general  skills  for  study  and  career  management,  articulate  their 
personal  goals  and  evaluate  progress  towards  their  achievement,  and  encourage  a positive 
attitude  to  learning  throughout  life. 

PDP  is  also  about  improving  and  encouraging  dialogue  between  learners  and  teachers. 
Discussion-based  seminars  offer  a structured  and  supported  PDP  process.  PDP  offers  another 
way  of  encouraging  students  to  think  about  what  they  know,  what  they  don't  know  and  how  they 
might  develop  the  skills  to  fill  the  gaps  in  their  knowledge  such  that  their  appreciation  of  the 
subject  area  improves.  It  also  enables  discussion  between  learners  and  other  parties,  and 
introduces  students  to  the  discipline  of  evidencing  and  documenting  work. 

1.3  Background  of  the  Reported  Work 

During  the  summer  2006,  the  College  of  Engineering  and  Engineering  Technology  (CEET)  of 
Northern  Illinois  University  took  an  initiative  towards  scholarship  of  teaching  and  learning.  The 
goal  of  this  initiative  was  to:  Institutionalize  and  sustain  a program  of  faculty  development  on 
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teaching,  student  learning,  and  educational  research  to  prepare  faculty  to  engage  in  scholarship 
of  teaching  through  action  research  in  the  classrooms.  As  a starting  of  this  initiative,  during  the 
summer,  a group  of  CEET  faculty  underwent  an  intensive  workshop  on  course  design  and 
development.  In  the  following  Eall  semester,  the  group  has  conducted  an  educational  research 
using  their  developed  courses.  The  author  is  a member  of  this  group  and  has  designed  and 
developed  and  introduced  an  introductory  level  digital  design  course  within  an  undergraduate 
program  and  subsequently  implemented  it  for  the  course. 

The  paper  will  present  the  course  design  and  development  part  of  the  activity  through 
reflective  practice  and  intentional  instructional  design.  In  the  process  the  author  has  studied 
various  teaching  models,  teaching  and  learning  styles,  taxonomy,  active  learning,  students 
learning  outcomes  and  their  relation  to  the  ABET  standards  and  learning  levels,  assessment 
planning,  item  development,  test  formulation,  and  test  analysis. 

The  next  section  describes  the  course  analysis  that  studies  various  fundamentals  of 
teaching  and  learning  theories  along  with  the  structure  of  target  course  design.  Section  three 
illustrates  a comprehensive  assessment  plan  by  highlighting  the  assessment  tools  that  are  being 
used  during  the  course  implementation  process.  Section  four  presents  the  objective  item 
development  and  test  formulation  process.  Section  five  discusses  the  test  analysis  and  its 
importance  to  maintain  the  quality  of  objective  tests.  These  are  followed  by  the  conclusions, 
acknowledgements,  and  references. 

2.  Course  Analysis 

This  section  involves  the  study  of  teaching  models,  teaching  and  learning  styles,  active  learning, 
and  bloom’s  traditional  and  revised  taxonomy  along  with  the  development  of  students  learning 
objectives,  content  outlines  and  course  priorities. 

2.1  Teaching  Models 

Before  moving  further  into  the  course  design,  it  may  be  good  idea  to  provide  with  a discussion 
on  teaching  models.  Teaching  models  describe  a learning  environment,  including  the  behavior 
of  the  teachers  when  that  model  is  used.  The  models  can  be  grouped  into  four  families,  whose 
members  share  orientations  towards  human  beings  and  how  they  learn:  a)  the  information 
processing  family;  b)  the  social  family;  c)  the  personal  family;  and  d)  the  behavioral  systems 
family  (Joyce  et  al,  2004) 

Information  processing  family:  emphasis  ways  of  enhancing  the  human  being’s  innate 
drive  to  make  sense  of  the  world  by  acquiring  and  organizing  data,  sensing  problems,  and 
generating  solutions  to  them,  and  developing  concepts  and  languages  for  conveying  them.  Some 
models  provide  the  learner  with  information  and  concepts,  emphasize  concept  formation  and 
hypothesis  testing,  and  other  generate  creative  thinking.  The  models  under  this  family  are- 
inductive  thinking,  concept  attainment,  scientific  inquiry,  inquiry  training,  mnemonics,  synectics, 
and  advance  organizers. 

Social  family:  are  constructed  by  building  learning  communities  that  generate  a collective 
energy  that  call  synergy.  The  models  under  this  family  are-  partner  in  learning,  group 
investigation,  role  play,  and  jurisprudential  inquiry. 

Personal  family:  pays  great  attention  to  the  individual  perspective  and  seeks  to  encourage 
productive  independence,  so  that  people  become  increasingly  self-aware  and  responsible  for  their 


Page  4 of  32 


Azad 


own  destinies.  The  models  under  this  family  are-  nondirective  teaching  and  enhancing  self- 
esteem. 


Behavioral  systems  family:  includes  programs  for  reducing  phobias,  learning  to  read  and 
compute,  developing  social  and  athletic  skills,  replacing  anxiety  with  relaxation,  and  learning  the 
complexes  of  intellectual,  social,  and  physical  skills 

2.2  Teaching  and  Learning  Styles 

Felder  presented  a discussion  on  teaching  and  learning  styles  (Felder,  1993).  They  presented  that 
students  learn  many  ways,  by  seeing  and  hearing,  reflecting  and  acting;  reasoning  logically  and 
intuitively,  memorizing  and  visualizing  and  drawing  analogies,  and  building  mathematical 
models.  At  the  same  time  teachers  also  use  various  styles  for  their  teaching,  such  as  lectures, 
demonstration,  leading  students  to  self-discovery,  principles,  application,  memory,  and  others  on 
understanding.  It  has  proven  that  increased  learning  gains  can  be  achieved  when  instructions  is 
designed  with  students’  learning  styles  in  mind  (Briggs,  1977).  Also,  attention  to  learning  styles 
and  learner  diversity  has  been  shown  to  increase  student  motivation  to  learn  (Hein  and  Budny, 
1999).  In  early  90’ s,  based  on  Carl  Jung’s  Dialectic  Tension,  and  Kurt  Lewin’s  Experimental 
Learning  Theory  Dr.  Kolb  devised  his  learning  style  inventory  (Kolb,  1985).  It  divides  students’ 
population  into  four  groups  depending  upon  what  a person  is  like,  as  opposed  to  reasoning  and 
thinking  skills.  The  categories  are:  type  1-  concrete,  reflective;  type-2  abstract,  reflective;  type-3 
abstract,  active;  and  type-4  concrete,  active.  An  objective  of  education  should  thus  be  to  help 
students  build  their  skills  in  both  their  preferred  and  less  preferred  modes  of  learning  by 
considering  various  teaching  and  learning  styles. 

2.3  Active  learning  and  Taxonomy  of  Learning  Analysis 

Students  learn  best  when  they  are  actively  involved  in  the  learning  process  and  the  experiences 
are  concrete  rather  than  abstract.  Active  learning  is  not  merely  a set  of  activities,  but  rather  an 
attitude  on  the  part  of  both  students  and  faculty  that  makes  learning  effective.  The  objective  of 
active  learning  is  to  stimulate  lifetime  habits  of  thinking  to  stimulate  students  to  think  about  how 
as  well  as  what  they  are  learning  and  to  increasingly  take  responsibility  for  their  own  education 
(Hatfield,  1995).  Students  whose  teachers  emphasize  higher-order  thinking  skills  and  hands-on 
learning  activities  outperform  their  peers  significantly.  Students  who  engage  in  hands-on 
learning  on  a weekly  basis  outperform  those  who  engage  in  this  manner  of  instruction  on  a 
monthly  basis.  Students  whose  teachers  conduct  hands-on  learning  activities  outperform  their 
peers  by  72%  of  a grade  level  in  math  and  40%  of  a grade  level  in  science.  This  study  indicates 
that  the  most  effective  classroom  practices  involve  conveying  higher  order  thinking  skills  and 
engaging  in  hands-on  learning  activities  (Educational  Testing  Service  2001). 

According  to  Dale’s  research  on  teaching  and  learning  there  is  a cone  of  leaning  which 
represent  the  level  of  learning  with  respect  to  the  type  of  activities  students  are  involved  with 
(Dale,  1969).  As  presented  in  Eigure  1,  the  least  effective  method,  the  top  of  the  cone,  involves 
learning  from  information  presented  through  verbal  symbols,  i.e.,  listening  to  spoken  words, 
while  the  most  effective  method,  the  bottom  of  the  cone  involves  direct,  purposeful  learning 
experiences,  such  as  hands-on  or  field  experiences. 
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Figure  1:  Dale’s  cone  of  learning  (developed  and  revised  by  Bruce  Hyland). 

The  cognitive  domain  involves  knowledge  and  the  development  of  intellectual  skills. 
This  includes  the  recall  or  recognition  of  specific  facts,  procedural  patterns,  and  concepts  that 
serve  in  the  development  of  intellectual  abilities  and  skills.  Bloom  created  a learning  taxonomy 
that  categorize  competency  level  in  the  cognitive  domain  (Bloom,  1956).  According  to  this, 
there  are  six  major  categories:  knowledge-  the  ability  to  remember  and  state  previously  learned 
materials,  comprehension-  the  ability  to  grasp  the  meaning  of  material  and  to  restate  it  in  one’s 
own  words,  application-  the  ability  to  use  learned  material  in  new  and  concrete  situations, 
analysis-  the  ability  to  break  down  material  into  its  components  so  as  to  understand  its 
organizational  structure,  synthesis-  the  ability  to  put  parts  together  to  form  a new  whole  system, 
and  evaluation-  the  ability  to  judge  the  value  of  material  for  a given  purpose.  The  categories  can 
be  thought  of  as  degrees  of  difficulties.  It  starts  from  the  simplest  behavior  as  knowledge  to  the 
complex  one  as  evaluation.  To  keep  the  importance  of  Bloom’s  work  relative  to  today’s 
theories,  Anderson  and  Krathwohl  (2001)  revised  Bloom’s  original  taxonomy  by  combining  both 
the  cognitive  process,  and  knowledge  dimensions.  This  new  expanded  taxonomy,  called 
Bloom’s  revised  taxonomy  that  help  instructional  designers  and  teachers  to  write  and  revise 
learning.  The  learning  levels  are:  creating-  putting  together  ideas  or  elements  to  develop  an 
original  idea  or  engage  in  creative  thinking;  evaluating-  judging  the  value  of  ideas,  materials  and 
methods  by  developing  and  applying  standards  and  criteria;  analyzing-  breaking  information 
down  into  its  component  elements;  applying-  using  strategies,  concepts,  principles  and  theories 
in  new  situations;  understanding-  understanding  of  given  information;  and  remembering-  recall 
or  recognition  of  specific  information. 

2.4  Student  Learning  Objectives 

A Student  learning  objectives  (SLO)  is  a statement  of  exactly  what  students  should  be  able  to  do 
after  completion  of  the  course  or  at  specified  points  during  the  course.  These  objectives  should 
be  measurable  and  must  use  an  action  verb  (e.g.,  such  as  define,  classify,  construct,  compute,  etc) 
rather  than  nebulous  verbs  reflecting  internal  states  that  cannot  be  observed  (e.g.,  known,  learn, 
understand,  realize,  and  appreciate).  Student’s  performance  verbs  by  level  of  cognitive  operation 
based  on  Bloom’s  Taxonomy  is  provided  below  (Nilson,  2003): 

Knowledge-  arrange,  order,  define,  recall  duplicate,  recite,  label,  recognize,  list,  relate,  memorize, 
repeat,  name,  and  reproduce. 
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Comprehension-  classify,  locate,  describe,  recognize,  discuss,  report,  explain,  restate,  express, 
review,  identify,  select,  indicate,  and  translate. 

Application-  apply,  interpret,  choose,  operate,  compose  examples,  practice,  demonstrate,  schedule, 
dramatize,  sketch,  employ,  solve,  illustrate,  and  use. 

Analysis-  analyze,  differentiate,  appreciate,  discriminate,  calculate,  distinguish,  categorize, 
examine,  compare,  experiment,  contrast,  question,  criticize,  and  test. 

Synthesis-  arrange,  integrate,  assemble,  manage,  collect,  organize,  compose,  plan,  construct, 
predict,  create,  prepare,  design,  propose,  formulate,  and  set  up. 

Evaluation-  appraise,  evaluate,  argue,  judge,  asses,  rate,  challenge,  score,  choose,  select,  defend, 
support,  dispute,  and  value. 

A total  of  seven  major  SLOs  have  developed  for  the  target  course,  and  all  these 
objectives  are  mapped  with  the  intended  teaching  styles,  learning  styles,  learning  model.  Dale’s 
learning.  Bloom’s  traditional  and  revised  taxonomy.  This  mapping  is  presented  as  a tabular  form 
in  Appendix-A  and  the  table  will  be  used  later  when  planning  for  assessment  strategies.  In  the 
table,  the  course  is  composed  of  fifteen  weeks  of  teaching  with  two  one  and  half  hours  of  class 
per  week. 

2.5  ABET  Outcomes  and  Student  Learning  Objectives 

As  an  ABET  (accreditation  boards  of  engineering  and  technology)  accredited  program,  it  is  a 
requirement  for  the  course  to  address  some  of  the  ABET  program  outcomes  that  are  based  on  the 
needs  of  the  program’s  constituencies.  The  measurable  outcomes  that  are  provided  by  the  ABET 
are  (ABET,  2007): 

(a)  An  ability  to  apply  knowledge  of  computing  and  mathematics  appropriate  to  the  discipline 

(b)  An  ability  to  analyze  a problem,  and  identify  and  define  the  computing  requirements 
appropriate  to  its  solution 

(c)  An  ability  to  design,  implement,  and  evaluate  a computer-based  system,  process,  component, 
or  program  to  meet  desired  needs 

(d)  An  ability  to  function  effectively  on  teams  to  accomplish  a common  goal 

(e)  An  understanding  of  professional,  ethical,  legal,  security  and  social  issues  and  responsibilities 

(f)  An  ability  to  communicate  effectively  with  a range  of  audiences 

(g)  An  ability  to  analyze  the  local  and  global  impact  of  computing  on  individuals,  organizations, 
and  society 

(h)  Recognition  of  the  need  for  and  an  ability  to  engage  in  continuing  professional  development 

(i)  An  ability  to  use  current  techniques,  skills,  and  tools  necessary  for  computing  practice. 

The  ABET  outcomes,  those  are  addressed  through  the  course  are  mapped  with  all  the 
SEOs.  At  the  same  time,  it  also  shows  which  Bloom’s  cognitive  process  level  will  be  achieved 
while  addressing  each  of  these  SEO.  Appendix-B  is  shows  the  mapping  of  SEOs  for  only  one  of 
the  ABET  outcomes. 

2.6  Content  Outline,  Course  Priorities,  and  Content  Schedule 

The  course  that  was  designed,  developed,  and  reported  is  an  introductory  level  digital  electronics 
course  within  an  undergraduate  electrical  engineering  technology  (EET)  program  of  Northern 
Illinois  University  (NIU).  After  the  study  of  all  the  theories  and  their  possible  use  for  the  course 
the  next  task  was  to  identify  the  course  outline  underlining  the  major  topics,  required  science 
foundation,  required  mathematics  foundation,  and  communication  foundation  skills.  Appendix- 
C shows  the  worksheet  with  all  these  information. 

Sometimes,  it  is  not  always  possible  to  address/deliver  all  the  listed  topics  within  a course 
schedule.  It  is  helpful  for  the  faculty  if  the  topics  can  be  identified  with  their  priority  levels.  The 
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priority  levels  ean  be  laid  out  into  three  stages:  essential,  possible  to  eseape,  and  not  essential. 
The  faculty  must  cover  the  essential  topics  and  try  to  cover  the  other  two  if  possible.  The  course 
outline  with  these  priority  levels  is  provided  in  Appendix-D. 

For  more  detailed  planning  a day-to-day  map  was  developed  with  each  course  topics 
along  with  corresponding  teaching  and  learning  models,  learning  styles,  bloom’s  learning  level, 
and  dales  cone  of  active  learning.  The  table  is  showing  in  Appendix-E. 

3.  Assessment  and  Evaluation  Plan 

Assessment  is  one  of  the  core  parts  of  the  course  development  activities.  Assessment  is  the 
process  of  gathering  and  discussing  information  from  multiple  and  diverse  sources  in  order  to 
develop  a deep  understanding  of  what  students  know,  understand,  and  can  do  with  their 
knowledge  as  a result  of  their  educational  experiences;  the  process  culminates  when  assessment 
results  are  used  to  improve  subsequent  learning  (Huba  and  Freed,  2000).  The  business 
community  axiom  that  What  gets  measured,  gets  done  holds  true  in  education  as  well.  There  are 
four  fundamental  elements  of  assessment  and  evaluation  process:  a)  Formulate  statements  of 
intended  learning  outcomes;  b)  develop  assessment  tools;  c)  implementation  of  tools;  and  d) 
evaluate  the  assessment  results  to  improve  learning. 


Figure  2:  Multifaceted  assessment  plan. 

A best  approach  is  to  use  a variety  of  assessment  tools  or  procedures  to  produce  a 
balanced  system  over  the  course  to  attain  the  identified  standards  (Scarborough,  2005).  A 
balance  system  should  include  good  and  reliable  traditional  assessment,  performance  assessment, 
and  other  methods.  It  is  important  to  include  as  many  assessment  tools  as  possible  to  provide 
accurate  and  useful  information  for  making  decisions  about  learning.  Objectively  scored  items 
are  prepared  to  focus  on  factual  knowledge  and  at  the  same  time  they  assess  knowledge  bit  by 
bit,  item  by  item,  typically  with  no  reference  to  any  eventual  real-world  application  (Resnick  and 
Resnick,  1992).  They  are  only  indirect  indicators  of  more  complex  abilities  such  as  reasoning 
about  cutting-edge  issues  or  using  information  to  solve  important  problems  in  a particular  field. 
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However,  the  ehallenges  faeed  by  adults  in  general  and  by  professionals  in  partieular  fields  tend 
to  be  those  that  require  the  simultaneous  coordination  and  integration  of  many  aspects  of 
knowledge  and  skill  in  situations  with  few  right  answers. 

The  aim  of  the  reported  course  development  process  was  to  develop  a multifaceted 
assessment  plan  by  including  a number  of  assessment  tools.  The  developed  assessment  plan  is 
shown  in  Figure  2.  The  assessment  plan  incorporates  five  different  kinds  of  assessment  tools:  a) 
observation;  b)  open-ended  tests;  c)  objective  tests;  d)  mapping;  and  e)  performance  assessment. 
A table  has  been  developed  to  map  how  these  assessment  methods  will  address  various  levels  of 
Bloom’s  learning  taxonomy  (Appendix-F)^ 

Observation:  is  the  planned  viewing  and  analysis  of  students,  their  work  environment, 
and  their  interactions  with  other  students,  and  their  teachers.  Observations  are  an  opportunity  to 
see  how  students  solve  problems  and  to  learn,  what  factors  may  affect  their  ability  to  learn, 
complete  work,  and  interact  in  a positive  way  with  others.  Observations  are  an  important  part  of 
the  special  education  diagnostic  assessment  process.  They  can  be  used  for  general  information 
gathering  or  designed  to  identify  specific  behaviors.  They  can  assess  the  student's  ability  to 
perform  specific  tasks  and  pinpoint  exactly  where  students  make  mistakes  in  their  work.  They 
can  be  unstructured  narratives,  semi-structured  forms,  or  highly  structured,  as  in  standardized 
behavior  checklists.  Within  the  proposed  assessment  plan  checklists,  logs,  and  rating  scale  are  to 
be  adopted  under  this  category. 

Open-ended:  is  the  subjective  type  of  assessment  tools.  To  assess  the  test  items  the 
faculty  needs  to  make  a personal  judgment  as  to  the  quality  of  the  response  e.g.  the  literary 
merits  of  an  essay  or  the  artistic  merits  of  a painting.  In  open-ended  assessments  it  usually  takes 
longer  to  judge  and  teacher's  judgment  is  one  of  the  main  factors  towards  grading.  These 
questions  are  lengthy  in  nature  and  take  longer  time  to  grade  than  objective  questions  and 
therefore  only  include  a small  number  of  questions,  focusing  on  complex  concepts.  Within  the 
proposed  assessment  plan,  descriptive  items  and  homeworks  are  used. 

Objective  test:  items  will  have  only  one  right  answer  so  there  is  no  human  factor  involved 
in  the  grading  process.  There  is  a range  of  item  types  that  one  can  choose  for  objective  test 
development.  Some  of  these  are:  Multiple  choice  questions  (MCQs)  are  the  traditional  ones 
where  students  choose  one  from  a list  of  possible  answers.  True/False  questions  require  a 
student  to  assess  whether  a statement  is  true  or  not.  Assertion-Reason  questions  combine 
elements  of  MCQ  and  true-false.  Multiple  response  questions  (MRQs)  are  similar  to  MCQs,  but 
involve  the  selection  of  more  than  one  answer  from  a list.  Matching  questions  involve  linking 
items  in  one  list  to  items  in  a second  list.  Ranking  questions  require  a student  to  relate  items  in  a 
column  to  one  another  and  can  be  used  to  test  the  knowledge  of  sequences,  order  of  events,  and 
level  of  gradation.  Sequencing  questions  require  the  student  to  position  text  or  graphic  objects  in 
a given  sequence.  These  are  particularly  good  for  testing  methodology.  For  the  proposed 
assessment  plan,  only  multiple  choice,  true-false,  and  matching  will  be  implemented.  However, 
other  kinds  of  items  are  also  a candidate  for  future  developments. 

Mapping:  It  has  been  established  that  the  essence  of  knowledge  is  structure  (Gasper  and 
Candaday,  2000;  Anderson,  1984).  Assuming  that  knowledge  within  a content  domain  is 
organized  around  central  concepts,  to  be  knowledgeable  in  the  domain  implies  a highly 
integrated  conceptual  structure  among  those  concepts.  As  expertise  in  a domain  grows,  through 
learning,  training,  and/or  experience,  the  elements  of  knowledge  become  increasingly 
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interconnected  (Chi,  et  al,  1988).  Considering  this  concept  maps  have  been  proposed  as  a more 
direct  approach  for  capturing  the  interrelatedness  among  concepts  in  a domain  (Ruiz-Primo  and 
Shavelson,  1996).  Concept  maps  have  been  shown  to  be  a viable  method  for  assessing  students’ 
understanding.  Concept  map  as  assessment  methodologies  have  been  developed  that  use  both 
closed-ended  concept  maps  (Ruiz-Primo  and  Shavelson,  1996)  and  open-ended  concept  maps 
(Abrams,  et  al,  2006).  In  the  proposed  assessment  plan  two  kinds  of  mappings  are  incorporated 
(hierarchy  and  flow  charting). 

Performance  assessment:  is  a measure  of  assessment  based  on  authentic  tasks  such  as 
activities,  exercises,  or  problems  that  require  students  to  show  what  they  can  do  (McBrien  and 
Brandt,  1997).  Some  performance  tasks  are  designed  to  have  students  demonstrate  their 
understanding  by  applying  their  knowledge  to  a particular  situation.  Performance  tasks  often 
have  more  than  one  acceptable  solution;  they  may  call  for  a student  to  create  a response  to  a 
problem  and  then  explain  or  defend  it.  The  process  involves  the  use  of  higher-order  thinking 
skills  (e.g.,  cause  and  effect  analysis,  deductive  or  inductive  reasoning,  experimentation,  and 
problem  solving).  Performance  tasks  may  be  used  primarily  for  assessment  at  the  end  of  a 
period  of  instruction,  but  are  frequently  used  for  learning  as  well  as  assessment. 

Performance  assessment  is  one  of  the  major  components  within  the  developed  course 
assessment  plan.  As  proposed  students  will  perform  three  performance  assessment  tasks 
throughout  the  duration  of  the  course.  Implementation  of  these  performance  assessments 
involves  research,  design  of  a system,  written  report  with  literature  survey,  oral  presentation,  and 
team  activities.  All  these  will  be  assessed  to  establish  the  level  of  learning  that  has  been 
achieved  by  the  students  in  terms  of  dealing  with  real-life  problems  within  the  given  subject  area. 

Sometime  it  is  difficult  for  students  to  understand  what  has  been  expected  by  the  faculty 
from  a performance  assessment  task.  This  can  causes  considerable  amount  of  misunderstanding 
between  the  students  and  a faculty  and  hinder  the  achievement  of  expected  level  of  expertise.  To 
handle  this  problem  it  is  important  to  develop  a rubric  while  writing  a performance  assessment. 

A rubric  is  a set  of  scoring  guidelines/criteria  for  evaluating  a given  performance 
assessment  task.  A rubric  answers  a number  of  questions:  a)  criteria  that  would  be  used  for 
judging;  b)  difference  between  good  work  and  weaker  work;  c)  present  the  judging  scores;  and 
d)  to  work  towards  excellence.  In  general,  rubrics  make  public  the  key  criteria  that  students  used 
in  developing,  judging,  and  revising  their  work  (Scarborough,  2005).  Rubrics  hold  both  the 
student  and  teacher  accountable.  Students  know  and  understand  what  they  have  to  do  to  achieve 
at  establish  levels,  and  teachers  cannot  change  the  rules  once  the  rubric  are  circulated.  Rubrics 
also  build  consistency  in  scoring  or  grading,  while  reducing  bias.  One  of  the  developed 
performance  assessment  along  with  its  rubrics  are  provided  in  Appendix-G. 

4.  Item  Development  and  Test  Formulation 

Item  development  is  a process  where  a faculty  develops  a range  of  test  items  for  each  learning 
outcomes  while  addressing  the  intended  level  of  expertise  as  identified  earlier.  The  goal  is  to 
maintain  a pool  of  examination  items  which  are  appropriate  to  measure  the  knowledge  and  skills 
necessary  for  effective  performance  in  the  field  of  practice  (Lunz,  2004).  There  is  a number  of 
issues  one  should  consider  while  developing  these  test  items:  a)  developing  new  items  on  a 
continuing  basis;  b)  reviewing  and  selecting  items  for  inclusion  in  the  written  examination;  c) 
monitoring  the  content,  task,  and  cognitive  skill  distributions  of  items;  d)  monitoring  the  content 
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quality  and  difficulty  of  each  item  and  avoiding  duplicate  items  on  the  same  knowledge/skill;  e) 
providing  expert  input  into  the  criterion  standard  against  which  candidates  are  measured;  and  f) 
reviewing  the  performance  of  each  item  to  ascertain  the  quality  of  the  content  and  structure  of 
the  item.  In  brief,  item  development  is  an  ongoing  process  where  a faculty  develops,  rewrite, 
modify,  and  delete  items  as  time  passes  by. 

The  reported  course  development  process  involves  test  development  only  for  the 
objectives  tests.  According  to  the  assessment  plan  the  items  to  be  developed  are  of  short  answer, 
multiple  choices,  true-false,  and  matching.  The  items  are  developed  for  each  of  the  students 
learning  objectives  while  addressing  each  learning  levels.  Table- 1 shows  a list  of  test  items  that 
have  been  developed  for  first  five  SLOs’  while  considering  identified  Bloom’s  level  of  learning. 


Table- 1:  Developed  test  items  for  SLOs. 


Student  Learning  Objectives 

Assessments;  Test 
Alignments 
Midterm  & Final 

Student  Learning  Objectives- 
Major 

student  learning  objectives  - minor 

Corresponding  Tests  and 
Test  Items 

1. 

To  examine  the  components  of  a 
digital  system. 

a 

To  contrast  between  analog  and 
digital  signals 

Multiole  choice:  laAl.  IbCl. 
IcCl,  lcC2,  IdKl,  ldK2,  leCl 

b 

To  classify  binary  digits,  logic  levels, 
and  digital  waveforms 

c 

To  compare  basic  logic  operations 

d 

To  categorize  fixed  function 
integrated  circuits 

e 

To  interpret  the  operation  of  simple 
digital  systems 

2. 

To  examine  the  structures  for 
various  number  systems. 

a 

To  distinguish  between  various  parts 
of  number  systems. 

Multiole  choice:  2aCl.  2aAl. 
2bCl,2bC2 

Short  answer:  2aKl.  2aK2,  2aK3, 
2aCl 

b 

To  examine  the  counting  in  binary, 
octal,  decimal,  and  octal. 

3. 

To  distinguish  the  conversion 
methods  for  various  number 
systems. 

a 

To  convert  between  binary  and 
decimal 

Multiole  choice:  3aAl.  3aA2. 
3aA3,  3aA4,  3aA5,  3bAl,  3bA2, 
3bA3,  3cAl,  3cA2 
Short  answer:  3cCl,  3bA2.  3bA3. 
3cAl,  3cA2 

b 

To  convert  between  binary  and 
hexadecimal 

c 

To  convert  between  binary  and  octal 

4. 

To  perform  different  binary 
arithmetic  operations: 
addition,  subtraction,  1 's 

complement,  2's  complement, 
and  signed  numbers. 

a 

To  examine  the  basic  rules  involving 
each  of  the  operations. 

Multiole  choice:  4aCl.  4aC2. 
4aC3,  4bAl,  4bA2,  4bA3,  4bA4. 
Short  answer:  4bAl.  4bA2,  4bA3, 
4bA4,  4aKl 

b 

To  use  the  rules  to  perform  each  of 
the  operations. 

5. 

To  examine  the  operation  and 
use  of  various  logic  gates  with 
different  input  patterns:  AND, 
OR,  and  NOT,  NAND,  NOR, 
XOR  and  XNOR. 

a 

To  develop  the  truth  tables  of  various 
logic  gates  using  established  rules. 

Multiple  choice:  5bCl.  5bC2, 
5cAl,  5dAl,  5dA2,  5cA2,  5dA4, 
5dA5. 

Short  answer:  5bCl.  5bC2.  5bC3. 
5bC4,  5bC5,  5bC6,  5dCl,  5dC2 

b 

To  use  the  truth  tables  to  identify 
output  pattern  of  a logic  gate  for  a 
given  set  of  input. 

c 

To  predict  output  logic  levels  for  a 
pulse  input  pattern. 

d 

To  recommend  the  use  of  appropriate 
logic  gate(s)  for  a given  application. 
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After  developing  a reasonable  amount  of  test  items  for  each  of  the  SLO,  it  is  now  possible  to 
formulate  a test  using  the  developed  test  items. 

5.  Test  Analysis 

For  traditional  test  methods,  such  as  open-ended  items  and  selected  response  items,  it  is 
important  to  assess  the  effectiveness  of  the  items  in  terms  of  the  objectives  of  the  assessment. 
Test  analysis  is  the  process  of  collecting,  summarizing,  and  using  information  from  students  test 
scores  to  make  decisions  about  each  item  (Nitko,  2004).  This  analysis  helps  a faculty  to: 
determine  whether  an  item  functions  as  it  was  intended,  feedback  to  student  about  their 
performance  and  as  a basis  for  classroom  discussion,  feedback  to  the  faculty  about  students’ 
difficulties,  areas  for  curriculum  development,  and  revising  the  assessment  tasks  and  improving 
item- writing  skills. 


Table  2:  Test  analysis  of  an  examination. 

Total  Across  Items 

Item  la  1b  1c  Id  2 3 4 5 6 7 (Student  Scores) 

Points  Possible  2 2 2 4 10  16  16  16  16  16  100  100% 


Students 


2 

2 

0 

4 

10 

14 

13 

13 

12 

10 

80 

80% 

1 

2 

2 

3 

10 

8 

8 

16 

8 

8 

66 

66% 

1 

2 

2 

4 

10 

10 

10 

10 

9 

8 

66 

66% 

1 

1 

1 

4 

10 

14 

16 

12 

10 

10 

79 

79% 

2 

2 

2 

4 

9 

16 

12 

16 

12 

16 

91 

91% 

2 

2 

2 

4 

8 

16 

12 

16 

16 

16 

94 

94% 

1 

2 

2 

1 

10 

13 

16 

15 

10 

11 

81 

81% 

2 

2 

2 

4 

10 

16 

8 

12 

10 

12 

78 

78% 

2 

0 

0 

0 

0 

0 

8 

6 

4 

4 

24 

24% 

0 

0 

0 

4 

8 

16 

6 

15 

10 

10 

69 

69% 

1 

1 

1 

4 

10 

15 

16 

16 

10 

16 

90 

90% 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0% 

2 

2 

0 

4 

10 

13 

16 

16 

16 

14 

93 

93% 

0 

0 

0 

4 

8 

12 

10 

10 

0 

0 

44 

44% 

2 

2 

0 

4 

10 

12 

9 

16 

10 

4 

69 

69% 

911 

70% 

Totals  Across  Students  (Item 
Scores) 


Max  Item  Scores 

26 

26 

26 

52 

130 

208 

208 

208 

208 

208 

1300 

Item  Score 

17 

18 

14 

40 

105 

151 

141 

163 

127 

135 

911 

Item  Difficulty 

65% 

69% 

54% 

77% 

81% 

73% 

68% 

78% 

61% 

65% 

70% 

Item  discrimination 

0.44 

0.71 

0.44 

0.81 

0.89 

0.92 

0.79 

0.93 

0.93 

0.94 

The  test  analysis  involves  few  steps:  a)  tabulating  all  the  details  about  the  test  items  (such 
as  students  details,  maximum  mark  for  each  items,  marks  attained  for  each  items);  b)  calculate 
the  difficulty  index  for  each  item;  and  c)  calculate  the  discrimination  index  for  each  item.  Item 
difficulty  index  is  the  fraction  of  the  total  group  answering  the  item  correctly.  Fraction  range  can 
vary  between  0.00  and  1.00.  When  the  fraction  tends  towards  1.00  indicates  that  the  item  was 
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easy  for  the  students.  It  is  for  the  individual  faculty  to  set  a desired  item  difficulty  for  a given 
course/examination  to  consider  a test  item  as  easy  or  difficult.  Item  discrimination  index  is  the 
relationship  (the  correlation)  between  the  students’  performance  on  the  items  and  the  student 
proficiency  in  the  content  the  item  measures.  The  item  should  discriminate  between  the 
proficient  students  and  the  non-proficient  students.  Proficient  students  should  do  well  on  the 
item,  non-proficient  students  should  not  do  well.  In  terms  of  measuring  student’s  proficiency, 
scores  on  the  entire  examination  is  considered  as  a measure  of  the  student  proficiency. 
Therefore,  the  item  discrimination  is  measured  as  the  correlation  between  students’  scores  on  an 
item  and  students’  scores  on  the  entire  examination.  As  a test  case,  an  item  analysis  is  presented 
for  a course  that  was  conducted  by  the  author  in  the  past  (Table-2).  Within  the  test  there  were  a 
total  of  ten  items,  with  an  aggregate  maximum  possible  item  score  can  be  1300,  where  students 
achieve  911.  Item  difficulty  for  individual  items  varies  between  54%  and  81%.  When  item 
discrimination  varies  between  0.44  and  0.94. 

6.  Discussion 

Usually,  almost  all  the  engineering  and  engineering  technology  faculty  starts  their  teaching 
carrier  with  the  highest  degree  in  their  professional  area.  Having  a terminal  qualification  in  the 
subject,  it  is  expected  that  one  can  teach  a course  with  full  authority.  There  is  a conception  that 
teaching  and  learning  is  a subject  area  of  education  discipline,  it  is  nothing  to  do  with 
engineering  and  engineering  technology.  However,  the  fact  is,  in  addition  to  knowledge  in  the 
subject  area  a teacher  should  use  appropriate  course  design  techniques,  proper  delivery  of  a 
course,  and  plan  for  a suitable  assessment  strategy.  The  reported  course  development  exercise 
following  an  intensive  teaching  and  learning  institute  provides  a valuable  lesson  for  the  author. 
Although,  the  author  was  using  some  of  the  techniques  from  his  prior  learning  from  various 
sources,  this  initiative  allows  him  to  think  a course  design  and  development  much  more 
comprehensive  manner,  while  considering  available  teaching  and  learning  theories  and  practices. 

7.  Conclusions 

The  paper  reports  the  design  and  development  of  an  introductory  level  digital  electronics  course 
for  an  undergraduate  EET  program.  The  process  has  been  divided  into  seven  stages:  (a)  study  of 
intentional  instructional  design  and  reflective  practices;  (b)  course  analysis  while  considering 
teaching  models,  teaching  and  learning  styles.  Dale’s  cone  of  learning  and  Bloom’s  traditional 
and  revised  taxonomy,  and  ABET  outcomes;  (c)  development  of  student  learning  objectives 
while  addressing  all  the  issues  mentioned  in  (b);  (d)  development  of  course  outline,  course 
priorities,  and  content  schedule;  (e)  development  of  an  multifaceted  assessment  and  evaluation 
plan;  (f)  Item  development  and  test  formulation;  and  (g)  test  analysis. 

Introduction  section  highlights  the  importance  of  intentional  instructional  design  and 
discusses  the  components  of  reflective  practices  along  with  their  benefits  in  attaining  course 
objectives.  A discussion  is  provided  on  the  available  teaching  models  and  teaching  and  learning 
styles.  Each  of  these  has  specific  purpose  and  reason  to  be  used  under  certain  circumstances 
during  a course  delivery.  Students  learn  different  ways  and  it  is  a good  idea  to  consider  as  many 
techniques  as  possible  to  address  the  need  of  majority  of  the  student  body. 

The  paper  also  discusses  the  Dale’s  cone  of  learning  along  with  Bloom’s  traditional  and 
revised  taxonomy.  Dale’s  cone  of  learning  illustrates  the  importance  of  active  learning  to 
achieve  higher  retention  of  knowledge,  while  Bloom’s  taxonomy  provides  classification  of  levels 
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of  intellectual  behavior  important  in  learning.  Bloom  identified  six  levels  within  the  cognitive 
domain,  from  the  simple  recall  or  recognition  of  facts,  as  the  lowest  level,  through  increasingly 
more  complex  and  abstract  mental  levels,  to  the  highest  order  which  is  classified  as  evaluation. 
For  course  design  process  it  is  important  for  a teacher  to  design  a course  with  attaining  highest 
level  as  possible  for  each  student  learning  objectives.  For  the  Bloom’s  revised  taxonomy  the 
former  six  categories  were  changed  from  noun  to  verb  forms.  The  reasoning  behind  this  is  that 
the  taxonomy  reflects  different  forms  of  thinking  and  thinking  is  an  active  process.  Another 
important  issue  considered  was  ABET  outcomes  that  is  a requirement  for  a course  within  an 
ABET  accredited  program. 

A set  of  student  learning  objectives  have  been  developed  while  addressing  all  the  issues 
involving  teaching  models,  teaching  and  learning  styles.  Dale’s  cone  of  learning.  Bloom’s 
traditional  and  revised  taxonomy.  This  is  one  of  the  major  steps  in  course  design  process.  This 
was  followed  by  development  of  course  outlines  with  various  pre-requirements,  course  priorities, 
and  content  schedule.  The  content  schedule  provides  a detailed  planning  with  day-to-day  map 
considering  each  course  topics  along  with  corresponding  teaching  and  learning  models,  learning 
styles,  bloom’s  learning  level,  and  Dale’s  cone  of  active  learning. 

Assessment  is  another  major  part  of  the  course  development  process  and  to  address  this 
issue  a comprehensive  assessment  plan  has  been  developed.  This  is  important  in  that  sense  that 
evaluating  students  learning  is  a major  and  one  of  the  most  difficult  parts  of  course 
implementation  process.  It  is  important  to  include  as  many  assessment  tools  as  possible  to 
provide  broader  picture  of  learning.  The  plan  includes:  observation,  open-ended  tests,  objective 
tests,  mapping,  and  performance  assessment. 

The  last  part  of  the  course  development  process  includes  the  development  of  performance 
assessment  projects,  test  item  development  for  objective  tests,  and  test  analysis.  Three 
performance  assessment  projects  have  been  developed  along  with  their  rubrics  that  are  to  be 
administrated  at  various  stages  of  the  course  delivery.  The  developed  items  for  the  objective 
tests  are  in  the  category  of  short  answer,  multiple  choices,  true-false,  and  matching.  The  test 
analysis  allows  a faculty  to:  determine  whether  an  item  functions  as  it  was  intended,  feedback  to 
student  about  their  performance  and  as  a basis  for  classroom  discussion,  feedback  to  the  faculty 
about  students’  difficulties,  areas  for  curriculum  development,  and  revising  the  assessment  tasks 
and  improving  item-writing  skills.  A sample  test  analysis  is  provided  to  demonstrate  the  process. 
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Appendix-A:  Content  Schedule  and  Styles,  Models,  Bloom’s  Analysis,  Dale’s  learning. 


Week 

Content  Topie 

Content 

Source 

Teaching 

Style 

Learning 

Style 

Teaching  Model 

Dale’s  Cone 

Bloom’s 

Traditional 

Bloom’s 

Revised 

1 

1 

1.  Number 
Systems 
Digital  and 
Analog 
Systems  (day- 
1) 

Number 
Systems  in  use 
(day-1) 
Number  base 
conversations 
(day-1) 

Text 

A and  B 

CR  and 
AR 

Progressive  Part 
Method,  Lecture, 
and  Graphic 
organizers 

Active  and 
Passive 

Knowledge 

Understand 

1 

Octal  and 
other  base 
systems  (day- 
2) 

Complements 

(day-2) 

Text 

A and  B 

CR  and 
AR 

Progressive  Part 
Method,  Lecture, 
and  Graphic 
organizers 

Active  and 
Passive 

Comprehension 

Understand 

2 

~T 

2.  Loaic  Gates 
Basic  Logic 
gates  (day-1) 
The  EXOR 
and  EXNOR 
gates  (day-1) 

Text 

A and  B 

CR  and 
AR 

Progressive  Part 
Method,  Lecture, 
and  Graphic 
organizers. 
Concept 
attainment 

Passive 

Comprehension 

Understand 

2 

Eixed  function 
logic  gates 
(day-2) 

Digital  logic 
IC  families 
(day-2) 
Realization  of 
various  gates 
from  different 
kinds  (2) 

Text 

A and  B 

CR,  AR, 
and  AA 

Lecture,  and 

Concept 

attainment 

Active  and 
Passive 

Application 

Apply 

3 

3.  Boolean 
Alsebra  and 
Loaic 

Simnlification 

Boolean 
operations  and 
expressions 
Laws  and  rules 
of  Boolean 
algebra 

Text 

A and  B 

AR 

Lecture,  Graphic 
organizers 

Active  and 
Passive 

Knowledge 

Understand 

3 

~7. 

De  Morgan’s 
Theorems 
Boolean 
analysis  and 
logic  circuits 
Standard  forms 
of  Boolean 
expressions 

Text 

A and  B 

AR  and 
CR 

Lecture,  Graphic 
organizers 

Active  and 
Passive 

Comprehension 

Understand 

4 

The  map 
method 
Two,  three, 
four  variable 
method 

Text  and 
Slides 

A,  B, and 
C 

AR  and 
AA 

Lecture  and 
Progressive  part 
method 

Active  and 
Passive 

Application 

Apply 

4 

Product  of 
Sum 

expression  and 
design 
Sum  of 
products  and 

Text 

A and  B 

AR  and 
CA 

Lecture,  Inquiry, 
and  Graphic 
organizers 

Active  and 
Passive 

Application 

Apply 
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design 

alternatives 

NAND-NOR 

implementatio 

ns 

5 

1 

4. 

Combinationa 
1 Losic 

Design 

Procedure 

Analysis 

techniques 

Text  and 
slides 

A and  B 

AR  and 
CA 

Lecture,  Inquiry, 
and  Graphic 
organizers 

Active  and 
Passive 

Analysis 

Analyze 

5 

Adders  (Half 
and  Full) 

Text  and 
slides 

A and  B 

AR  and 
CA 

Lecture,  Inquiry, 
and  Graphic 
organizers 

Active  and 
Passive 

Application 

Analyze 

6 

~T 

Binary  adder 
Carry  look 
ahead  adder 

Text  and 
slides 

A and  B 

AR  and 
AC 

Lecture,  Inquiry, 
and  Graphic 
organizers 

Passive 

Application 

Apply 

6 

Binary 
subtractor 
Decimal  adder 

Text  and 
slides 

A and  B 

AR  and 
AC 

Lecture,  Inquiry, 
and  Graphic 
organizers 

Passive 

Application 

Apply 

7 

Binary 

multiplier 

Text  and 
slides 

A and  B 

AR  and 
AC 

Lecture,  Inquiry, 
and  Graphic 
organizers 

Passive 

Application 

Apply 

77 

Magnitude 

comparator 

Text  and 
slides 

A and  B 

AR  and 
AC 

Lecture,  Inquiry, 
and  Graphic 
organizers 

Active  and 
Passive 

Application 

Apply 

8 

~T 

5.  MSI  and 
PLD 

components 

Decoders  and 
encoders 

Text  and 
slides 

A and  B 

AR  and 
CA 

Lecture,  Inquiry, 
and  Graphic 
organizers 

Active  and 
Passive 

Application 

Apply 

8 

Multiplexers 

and 

demultiplexers 

Text  and 
slides 

A and  B 

AR  and 
CA 

Lecture,  Inquiry, 
and  Graphic 
organizers 

Active  and 
Passive 

Application 

Apply 

9 

Programmable 
Logic  Array 

Text 

A 

AR  and 
CR 

Lecture  and 
Graphic  organizers 

Active  and 
Passive 

Comprehension 

Understand 

9 

Multilevel 
gates  and  their 
use  in  design 
procedures 

Text  and 
slides 

A and  B 

AR  and 
CA 

Lecture,  Inquiry, 
and  Graphic 
organizers 

Active  and 
Passive 

Application 

Apply 

10 

6.  Seauential 
Loaic 

Flip-Flops 
(SR,  JK,  D,  T) 

Text  and 
slides 

A and  B 

CR,  AR, 
and  AC 

Lecture,  Inquiry, 
and  Graphic 
organizers 

Active  and 
Passive 

Comprehension 

Understand 

10 

Flip-Flops 
(SR,  JK,  D,  T) 

Text  and 
slides 

A and  B 

CR  and 
AC 

Lecture,  Inquiry, 
and  Graphic 
organizers 

Active  and 
Passive 

Application 

Apply 

11 

~T 

Characteristic 
equations  and 
excitation 
tables 

Text  and 
slides 

A and  B 

CR,  AR, 
and  AC 

Lecture,  Inquiry, 
and  Graphic 
organizers 

Active  and 
Passive 

Analysis 

Analyze 

11 

Clocked 

sequential 

circuits 

Text  and 
slides 

A and  B 

AR  and 
AC 

Lecture,  Inquiry, 
and  Graphic 
organizers 

Active  and 
Passive 

Application 

Apply 

12 

~P 

State  reduction 
assignment 

Text  and 
slides 

A and  B 

CR  and 
AC 

Lecture,  Inquiry, 
and  Graphic 
organizers 

Active  and 
Passive 

Application 

Apply 
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12 

2 

State  reduction 
assignment 

Text  and 
slides 

A and  B 

CR  and 
AC 

Lecture,  Inquiry, 
and  Graphic 
organizers 

Active  and 
Passive 

Application 

Apply 

13 

7.  ADDlication 
of  Seauential 
Logic 

Design  of 
counters  using 
flip-flops 

Text  and 
slides 

A,  B, and 
C 

CR  and 
CA 

Lecture,  Inquiry, 
and  Graphic 
organizers 

Active  and 
Passive 

Application 

Apply 

13 

Design  of 
counters  using 
flip-flops 

Text  and 
slides 

A,  B, and 
C 

CR  and 
CA 

Lecture,  Inquiry, 
and  Graphic 
organizers 

Active  and 
Passive 

Analysis 

Analyze 

14 

~T 

Registers 

Text  and 
slides 

A,  B, and 
C 

CR  and 
CA 

Lecture,  Inquiry, 
and  Graphic 
organizers 

Active  and 
Passive 

Application 

Apply 

14 

Counters 

Text  and 
slides 

A,  B, and 
C 

CR  and 
CA 

Lecture,  Inquiry, 
and  Graphic 
organizers 

Active  and 
Passive 

Application 

Apply 

15 

Memory 
systems  (RAM 
and  ROM) 

Text  and 
slides 

A,  B, and 
C 

CR  and 
CA 

Lecture,  Inquiry, 
and  Graphic 
organizers 

Active  and 
Passive 

Comprehension 

Understand 

15 

Memory 

decoding 

Text  and 
slides 

A,  B, and 
C 

CR  and 
CA 

Lecture,  Inquiry, 
and  Graphic 
organizers 

Active  and 
Passive 

Knowledge 

Understand 
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Appendix-B:  Mapping  of  SLOs  with  Bloom’s  cognitive  process  dimension 


ABET/TAC 

Bloom's 

Learning 

Knowledge 

Outcomes 

Dimension 

A.  Mastery  of 

Factual 

knowledge, 

techniques. 

Knowledge 

skills,  modern 

Conceptual 

tools  of 

Knowledge 

disciplines. 

Procedural 

Knowledge 

Meta- 

Cognitive 

Knowledge 

Student  Learning  Objectives 


Bloom’s  Cognitive  Process  Dimension 


Knowledge 

Remember 


Comprehension 

Understand 


Application 

Apply 


Analysis 

Analyze 


Synthesis 

Evaluate 


Evaluate 

Create 


1.  To  examine  the  components  of  a digital 
system. 

a.  To  contrast  between  analog  and  digital  signals 

b.  To  classify  binary  digits,  logic  levels,  and 
digital  waveforms 

c.  To  compare  basic  logic  operations 

d.  To  categorize  fixed  function  integrated  circuits 

e.  To  interpret  the  operation  of  simple  digital 
systems 

2.  To  examine  the  structures  for  various  number 
systems. 

a.  To  distinguish  between  various  parts  of  number 
systems. 

b.  To  examine  the  counting  in  binary,  octal, 
decimal,  and  octal. 

3.  To  distinguish  the  conversion  methods  for 
various  number  systems. 

a.  To  convert  between  binary  and  decimal 

b.  To  convert  between  binary  and  hexadecimal 

c.  To  convert  between  binary  and  octal 

4.  To  perform  different  binary  arithmetic 
operations:  addition,  subtraction,  I's 
complement,  2's  complement,  and  signed 
numbers. 

a.  To  examine  the  basic  rules  involving  each  of  the 
operations. 

b.  To  use  the  rules  to  perform  each  of  the 
operations. 

5.  To  examine  the  operation  and  use  of  various 


1 


1 
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ABET/TAC 

Learning 

Outcomes 

Bloom's 

Knowledge 

Dimension 

Student  Learning  Objectives 

Bloom’s  Cognitive  Process  Dimension 

Knowledge 

Remember 

Comprehension 

Understand 

Application 

Apply 

Analysis 

Analyze 

Synthesis 

Evaluate 

Evaluate 

Create 

logic  gates  with  different  input  patterns:  AND, 
OR,  and  NOT,  NAND,  NOR,  XOR  and  XNOR. 

a.  To  develop  the  truth  tables  of  various  logic 
gates  using  established  rules. 

b.  To  use  the  truth  tables  to  identify  output  pattern 
of  a logic  gate  for  a given  set  of  input. 

c.  To  predict  output  logic  levels  for  a pulse  input 
pattern. 

d.  To  recommend  the  use  of  appropriate  logic 
gate(s)  for  a given  application. 

6.  To  analyze  the  properties  of  fixed-function 
logic  integrated  circuits  (IC):  Complementary 
Metal  Oxide  Semiconductor  (CMOS)  and 
Transistor-Transistor  Logic  (TTL). 

a.  To  identify  various  supply  voltage  and  power 
requirements  for  CMOS  and  TTL  ICs. 

b.  To  analyze  the  generic  numbering  convention 
for  CMOS  and  TTL  ICs. 

c.  To  classify  common  logic  gate  ICs  according  to 
their  standard  identifier  digit. 

d.  To  examine  the  logic  gate  configuration  within 
an  IC. 

e.  To  compare  alternative  logic  symbols  for 
representing  logic  gates  while  drawing  a circuit 
diagram. 

f To  examine  the  voltage  values  for  input  output 
logic  levels  for  CMOS  and  TTL  ICs. 

8.  To  use  Laws  and  Rules  of  Boolean  algebra 
and  DeMorgan’s  Theoerms  for  manipulating 
Boolean  expressions. 

a.  To  use  the  commutative,  associative,  and 
distributive  laws  to  manipulate  Boolean 
expressions. 

1 

1 
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ABET/TAC 

Learning 

Outcomes 

Bloom's 

Knowledge 

Dimension 

Student  Learning  Objectives 

Bloom’s  Cognitive  Process  Dimension 

Knowledge 

Remember 

Comprehension 

Understand 

Application 

Apply 

Analysis 

Analyze 

Synthesis 

Evaluate 

Evaluate 

Create 

b.  To  examine  the  use  of  Boolean  rules  while 
manipulating  Boolean  expressions. 

c.  To  use  DeMorgan’s  Theorems  for  manipulating 
Boolean  expressions. 

d.  To  adapt  the  Boolean  laws,  Boolean  rules,  and 
DeMorgan’s  Theorems  while  minimizing  Boolean 
expressions. 

9.  To  analyze  digital  logic  circuits  using  Boolean 
algebra. 

a.  To  develop  a Boolean  expression  for  a given 
logic  circuit. 

b.  To  evaluate  a Boolean  expression  and  prepare  a 
truth-table  for  the  logic  circuit. 

c.  To  demonstrate  the  use  of  Boolean  algebra 
while  minimizing  Boolean  expressions. 

12.  To  minimize  logic  expressions  using 
Karnaugh  map  (K-map). 

a.  To  develop  K-maps  with  different  size  of  input 
variables  (1  to  4). 

b.  To  map  SOP  expressions  on  K-maps. 

c.  To  develop  minimized  expressions  from  K- 
maps.. 

d.  To  construct  K-map  from  a non-structured  SOP 
expression. 

15.  To  evaluate  the  properties  of  Latches,  Flip- 
Flops,  and  timers. 

a.  To  contrast  between  Latches  and  Flip-Flops. 

b.  To  evaluate  the  properties  of  edge-triggered  J-K 
Flip-Flop. 

c.  To  evaluate  the  properties  of  edge-triggered  D 
Flip-Flop. 

d.  To  evaluate  the  properties  of  edge-triggered  S-R 
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ABET/TAC 

Learning 

Outcomes 

Bloom's 

Knowledge 

Dimension 

Student  Learning  Objectives 

Bloom’s  Cognitive  Process  Dimension 

Knowledge 

Remember 

Comprehension 

Understand 

Application 

Apply 

Analysis 

Analyze 

Synthesis 

Evaluate 

Evaluate 

Create 

Flip-Flop. 

e.  To  utilize  the  asynchronous  Preset  and  Clear 
inputs  of  Flip-Flops. 

f.  To  examine  the  operating  characteristics  of  Flip- 
flops,  such  as-  propagation  delay  times,  set-up 
time,  hold  time.  Maximum  clock  frequency.  Pulse 
width,  and  Power  dissipation. 

g.  To  compare  the  properties  of  commercially 
available  Flip-Flop  ICs. 
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Appendix-C:  Prerequisites  in  terms  of  science  foundation,  mathematics  foundation,  and 
communication  skills. 

SLOs  Pre-requisite 


Science(s)  Foundation 
Required 

Mathematics 

Foundation 

Required 

Communication 
Foundation/  Skills 
Required 

1.  Number  Systems 

Digital  and  Analog  Systems 
Number  Systems  in  use 
Number  base  conversations 
Octal  and  other  base  systems 
Complements 

Electricity  and  magnetism 
Semiconductor  and  its 
properties  in  relation  to 
electronics. 

Basic  algebra 

Ability  to  comprehend, 
analyze,  and  interrogate 
critically 

2.  Logic  Gates 

Basic  Logic  gates 
The  EXOR  and  EXNOR  gates 
Eixed  function  logic  gates 
Digital  logic  IC  families 
Realization  of  various  gates  from 
different  kinds 

Series  and  parallel  circuit 

philosophy 

Laws  and  theorems 

involving  electricity  and 

magnetism 

Semiconductor  and  its 
properties  in  relation  to 
electronics. 

Basic  algebra 
Logic 

Ability  to  comprehend, 
analyze,  and  interrogate 
critically 

3.  Boolean  Algebra  and  Logic 
Simplification 

Boolean  operations  and  expressions 

Laws  and  rules  of  Boolean  algebra 

De  Morgan’s  Theorems 

Boolean  analysis  and  logic  circuits 

Standard  forms  of  Boolean 

expressions 

The  map  method 

Two,  three,  four  variable  method 

Product  of  Sum  expression  and 

design 

Sum  of  products  and  design 
alternatives 

NAND-NOR  implementations 

Linear  algebra 

Quantitative  and 
qualitative  reasoning 
Use  of  resources 

4.  Combinational  Logic 

Design  Procedure 
Analysis  techniques 
Adders  (Half  and  Pull) 
Binary  adder 
Carry  look  ahead  adder 
Binary  subtractor 
Decimal  adder 
Binary  multiplier 
Magnitude  comparator 

Linear  algebra 

Quantitative  and 
qualitative  reasoning 
Use  of  resources  including 
technology. 

5.  MSI  and  PLD  components 

Decoders  and  encoders 
Multiplexers  and  demultiplexers 
Programmable  Logic  Array 
Multilevel  gates  and  their  use  in 
design  procedures 

Digital  signal  properties 

Linear  algebra 

Quantitative  and 
qualitative  reasoning 
Use  of  resources  including 
technology. 
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6.  Sequential  Logic 

Flip-Flops  (SR,  JK,  D,  T) 
Characteristic  equations  and 
excitation  tables 
Clocked  sequential  circuits 
State  reduction  assignment 

Digital  signal  properties 

Quantitative  and 
qualitative  reasoning 
Use  of  resources  including 
technology. 

7.  Application  of  Sequential  Logic 

Design  of  counters  using  flip-flops 

Registers 

Counters 

Memory  systems  (RAM  and  ROM) 
Memory  decoding 

Digital  signal  properties 

Quantitative  and 
qualitative  reasoning 
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Appendix-D:  Course  topics  with  priority  levels. 

Note:  E-  essential;  P-  possible  to  escape;  NE-  not  essential 

1.  Digital  Concepts 

Understanding  of  digital  systems  (E) 

Binary  digits,  logic  levels,  and  digital  waveforms  (E) 

Basic  logic  operations  (E) 

Eixed  function  ICs  (E) 

Digital  system  applications  (E) 

2.  Number  Systems 

Anatomy  of  number  systems  and  different  number  systems  (E) 

Number  base  conversations  between  common  number  systems  (E) 

- Binary,  octal,  decimal,  and  hexadecimal 
Complements-  2s  (P) 

Binary  addition  (E) 

Binary  subtraction  (P) 

3.  Logic  Gates 

Concept  of  logic  gates  (E) 

- Basic  Logic  gates  (AND,  OR,  NOT,  NAND,  NOR)  (E) 

- EXOR  and  EXNOR  gates  (E) 

Eixed  function  logic  gates  and  digital  logic  IC  families  (E) 

Basic  characteristics  of  IC  families  (P) 

Realization  of  various  gates  with  other  kinds  (NE) 

4.  Boolean  Algebra  and  Logic  Simplification 

Boolean  operations  and  expressions  (E) 

Laws  and  Rules  of  Boolean  algebra  (E) 

De  Morgan’s  Theorems  (E) 

Boolean  expressions  for  logic  circuits  (E) 

Minimization  of  Boolean  expressions  using  Laws  and  Rules  of  Boolean  algebra  and  De  Morgan’s 
Theorems.  (P) 

Minimization  of  Boolean  expressions  using  K-map  method  (upto  4 variables)  (E) 

Standard  forms  of  Boolean  expressions  (P) 

Product  of  Sum  expression  and  design  (P) 

Sum  of  products  expression  and  design  alternatives  (NE) 

5.  Combinational  Logic  Analysis 

Basic  combinational  logic  circuits  (E) 

Concept  of  digital  systems  design  (E) 

Detailed  design  procedure  (E) 

Analysis  of  digital  systems  (E) 

Universal  property  of  NAND  and  NOR  gates  (P) 

Logic  operations  with  pulse  inputs  (NE) 

6.  Functions  of  Combinational  Logic 

- Adders  (Half  and  Lull)  (E) 

Carry  look  ahead  adder  (NE) 

Binary  subtractor  (NE) 

Binary  multiplier  (P) 

Magnitude  comparator  (E) 

Decoders  and  encoders  (E) 

Multiplexers  and  demultiplexers  (E) 

Programmable  Logic  Array  (NE) 

Multilevel  gates  and  their  use  in  design  procedures  (NE) 
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7.  Latches,  Flip-Flops,  and  Timers 

Latches  (E) 

- Flip-Flops  (SR,  JK,  D,  T)  (E) 

Characteristic  equations  and  excitation  tables  for  each  types  (E) 
Clocked  sequential  circuits  (E) 

Flip-flop  applications  (E) 

One  shot  (NE) 

- The  555  timer  (NE) 

8.  Counters 

Asynchronous  counters  (E) 

Synchronous  counters  (E) 

Up-down  counters  (NE) 

Counter  applications  (E) 

9.  Shift  Registers 

Basic  shift  register  functions  (E) 

Serial  In/Serial  out  shift  registers  (E) 

Serial  In/Parallel  out  shift  registers  (NE) 

Parallel  In/Serial  out  shift  registers(NE) 

Parallel  In/Parallel  out  shift  registers  (E) 

Bidirectional  shift  registers  (NE) 

Shift  register  applications  (E) 
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Appendix-E:  Course  topics,  Teaching  model  and  Teaching  styles,  Learning  styles.  Bloom’s  knowledge  level.  Dale’s  learning,  and 
topics. 


Week 

Obj 

TM 

TS 

LS 

B 

D 

Topics/Lab 
Activities  & 
Due  Dates 

TM 

TS 

LS 

B 

D 

Topics/Lab 
Activities  & Due 
Dates 

1 

8/28 

Inductive 
thinking  / F 

Assimil 

ating 

Comprehensi 

on 

Digital  Concepts 

Direct  instruction 
and  inductive 
thinking/F 

Assimilat 

ing 

Application 

Receiving 

/participating 

Number  Systems 

2 

9/4 

Concept 
attainment , and 
Direct  instruction 
/B,F 

Assimil 

ating 

Comprehensi 

on 

Receiving  / 
participating 

Number  Systems 

Concept  attainment 
and  direct 
instruction  / B,  F 

Assimilat 

ing 

Comprehen 

sion 

Receiving  / 
participating 

Logic  Gates 

■ 

Concept 
attainment,  and 
Direct  instruction 
/A 

Assimil 

ating 

Application 

Receiving  / 
Participating 

Logic  Gates 

Concept  attainment 
and  direct 
instruction  / B 

Assimilat 

ing 

Comprehen 
sion  / 

application 

Receiving  / 
Participating 

Logic  Gates 

4 

9/18 

Inductive 
thinking  / B,  D 

Assimil 

ating 

Application 

Receiving  / 
Participating 

Logic  Gates 

Direct  instruction. 
Content  attainment, 
and  Partners  in 
learning  / A,  B,  H 

Assimilat 

ing/ 

Diverging 

Comprehen 

sion 

Receiving  / 
Participating 

Boolean  Algebra  and 
Logic  Simplification 

Performance  Test-1 

5 

9/25 

Content 
attainment,  and 
Inductive 
thinking  / B,  D 

Assimil 

ation 

Application 

Receiving  / 
Participating 

Boolean  Algebra 
and  Logic 
Simplification 

Direct  instruction, 
and  Content 
attainment  / A,  B 

Assimilat 

ing 

Comprehen 

sion 

Receiving  / 
Participating 

Combinational  Logic 
Analysis 

6 

10/2 

Inductive 
thinking,  and 
Content 

attainment  / B,  D 

Assimil 

ation 

Application 

Receiving  / 
Participating 

Combinational 
Logic  Analysis 

Inductive  thinking, 
and  Content 
attainment  / B,  D 

Assimilat 

ion 

Application 

Doing 

Combinational  Logic 
Analysis 

7 

10/9 

Direct 

instruction,  and 
Inductive 
thinking  / A,  B 

Assimil 

ating 

Comprehensi 

on 

Visual  receiving 

Functions  of 

Combinational 

Logic 

Inductive  thinking, 
and  Content 
attainment  / B,  D 

Assimilat 

ion 

Application 

Receiving  / 
Participating 

Functions  of 
Combinational  Logic 

8-10/16 

MX 

Mnemonics, 
Inductive 
thinking,  and 
Content 

attainment  / B,  D 

Assimil 

ation 

Application 

Receiving  / 
Participating 

Functions  of 

Combinational 

Logic 

MIDTERM 
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9 

10/23 

Inductive 
thinking  / B,  D 

Assimil 

ation 

Application 

Doing 

10 

10/30 

Direct 

instruction,  and 
Inductive 
thinking  / A,  B 

Assimil 

ating 

Comprehensi 

on 

Visual  receiving 

11 

11/6 

Content 
attainment  and 
Inductive 
thinking  / B,  D 

Assimil 

ation 

Application 

Receiving  and 
participating 

■ 

Inductive 
thinking  and 
Content 

attainment  / B,  D 

Assimil 

ation 

Comprehensi 

on 

Receiving  and 
Participating 

Direct  instruction 
and  Content 
attainment  / A,  B 

Assimil 

ation 

Comprehensi 

on 

Visual 

Receiving  and 
Participating 

14 

11/27 

Inductive 
thinking  and 
Content 

attainment  / B,  D 

Assimil 

ation 

Application 

Receiving  and 
Participating 

Inductive 
thinking  and 
Content 

attainment  / B,  D 

Assimil 

ation 

Application 

Receiving  and 
Participating 

16-2/11 

FE 

Inductive 
thinking  and 
Content 

attainment  / B,  D 

Assimil 

ation 

Application 

Receiving  and 
Participating 

Functions  of 

Combinational 

Logic 


Latches,  Flip- 
Flops,  and  Timers 


Latches,  Flip- 
Flops,  and  Timers 


Counters 


Shift  Registers 


Shift  Registers 


Shift  Registers 


Shift  Registers 
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Inductive  thinking, 
Partners  in  learning. 
Group  investigation  / 
B,  D,H 

Assimilat 
ion  / 

Diverging 

Application 

Doing 

Functions  of 
Combinational  Logic 

Performance  Test-2 

Content  attainment 
and  Indnctive 
thinking  / B,  D 

Assimilat 

ion 

Application 

Receiving  and 
participating 

Latches,  Flip-Flops,  and 
Timers 

Direct  instruction 
and  Content 
attainment  / A,  B 

Assimilat 

ion 

Comprehen 

sion 

Visual  Receiving 
and  Participating 

Counters 

Indnctive  thinking 
and  Content 
attainment  / B,  D 

Assimilat 

ion 

Application 

Visual  Receiving 
and  Participating 

Counters 

11/23 

THANKSGIVING 

Indnctive  thinking 
and  Content 
attainment  / B,  D 

Assimilat 

ion 

Application 

Visual  Receiving 
and  Participating 

Shift  Registers 

Indnctive  thinking. 
Partners  in  learning. 
Group  investigation  / 
B,  D,H 

Assimilat 
ion  / 

Diverging 

Application 

Doing 

Performance  Test-3 

Final  Examination 
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Appendix-F:  Bloom’s  Learning  Taxonomy  - Analysis  Chart  for  Assessment  Components. 


Assessment 

Knowledge 

Comprehension 

Application 

Analyze 

Synthesize 

Evaluate 

Remember 

Understand 

Apply 

Analyze 

Evaluate 

Create 

Midterm 

X 

X 

X 

8% 

38% 

54% 

Final 

X 

X 

X 

4% 

35% 

61% 

Performance  test-1 

X 

X 

50% 

50% 

Performance  test-2 

X 

X 

50% 

50% 

Performance  test-3 

X 

X 

50% 

50% 

Homeworks 

X 

X 

X 

40% 

30% 

30% 

Concept  map 

X 

X 

40% 

60% 
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Appendix- G:  Performance  Assessment  Project. 


Content  Standard 

ABET  Outcomes: 

A.  Mastery  of  knowledge,  techniques,  skills,  modern  tools  of  disciplines. 

B.  Apply  current  knowledge  and  adapt  to  emerging  applications  of  math,  science,  engineering,  and 
technology. 

D.  Ability  to  apply  creativity  in  the  design  of  systems,  components,  or  processes  appropriate  to 
program  objectives. 

E.  Function  effectively  on  teams. 

F.  Identify,  analyze,  and  solve  technical  problems. 

G.  Communicate  effectively  in  writing. 

H.  Communicate  effectively  orally. 

I.  Recognize  the  need  for,  and  an  ability  to  engage  in  life  long  learning 

L.  Commitment  to  quality,  timeliness,  and  continuous  improvement. 

Student  Learning  Objectives 

1.  To  examine  the  components  of  a digital  system. 

5.  To  examine  the  operation  and  use  of  various  logic  gates  with  different  input  patterns:  AND,  OR, 
and  NOT,  NAND,  NOR,  XOR  and  XNOR. 

6.  To  analyze  the  properties  of  fixed-function  logic  integrated  circuits  (IC):  Complementary  Metal 
Oxide  Semiconductor  (CMOS)  and  Transistor-Transistor  Logic  (TTL). 

7.  To  analyze  the  performance  characteristics  and  parameters  for  logic  gates  and  evaluate  their 
significance  in  digital  design. 

8.  To  use  Laws  and  Rules  of  Boolean  algebra  and  DeMorgan’s  Theoerms  for  manipulating  Boolean 
expressions. 

9.  To  analyze  digital  logic  circuits  using  Boolean  algebra. 

12.  To  minimize  logic  expressions  using  Karnaugh  map  (K-map). 

13.  To  analyze  digital  systems  using  combinational  logic. 

14.  To  evaluate  combinational  logic  circuits  for  commonly  used  digital  functionalities:  Half-adders 
and  full-adders,  parallel  binary  adders,  comparators,  BCD  to  decimal  decodes,  BCD  to  7-segment 
decoders,  encoders,  multiplexers,  and  demultiplexers. 

15.  To  evaluate  the  properties  of  Latches,  Flip-Flops,  and  timers. 

18.  To  design  and  study  of  counter  applications  using  Flip-Flops. 

1.  Problem  Statement 

You  are  working  with  an  industry  that  produces  pharmaceutical  pills.  At  some  point,  you  are  assigned 
to  design  a digital  controller  that  will  perform  the  following  tasks: 

a)  Differentiate  pills  from  one  kind  to  another  and  direct  them  towards  different  parts  of  the  facility 

b)  Count  the  pills  of  each  type  and  bottle  them  with  different  bottle  counts 

[The  problem  statement  provides  a general  description  of  the  system.  You  need  to  assume  other  details 
when  developing  the  concept  designs.  Make  sure  to  specify  the  assumptions  at  the  very  beginning  of  the 
project.] 

2.  Task  Description 

During  the  project  student  needs  to  perform  the  following  tasks: 

2.1  Develop  a number  of  concept  designs  with  the  given  problem  statement  {section  1 and  your  own 
assumptions). 

2.2  Pick  one  of  the  concept  designs  as  your  final  design  choice  and  elaborate  its  working  functionalities 
with  details. 
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2.3  Develop  detailed  design  and  identify  generic  components  that  need  to  be  used  for  the  project 
implementation. 

2.4  Research  on  identified  components  and  justify  their  selection  considering  the  design  factors  (provided 
in  section  3)  along  with  the  problem  statement  and  your  own  assumptions. 

2.5  Complete  the  design  with  selected  components  and  develop  a complete  drawing  along  with  a 
description  of  its  operation,  [please  keep  in  mind  that  your  design  should  contain  all  the  information  so 
that  one  can  assemble  the  system  without  any  enquiry] 

2.6  Evaluate  the  design  in  terms  of  achieving  the  identified  functionalities  [section  2.2]. 

2.7  Propose  alternative  designs/modifications  that  can  be  implemented  to  improve  the  current  design. 

3.  Design  factors  that  should  be  considered  during  the  design  process 

The  designing  the  project  you  should  consider  the  following  criteria: 

a)  Lowest  cost  possible 

b)  Minimum  power  consumption  while  fulfilling  the  design  assumptions 

c)  Highest  possible  speed  of  operation  as  required  by  the  system 

d)  Smallest  physical  size 

e)  System  should  stand  assumed  environmental  conditions 

f)  System  should  be  designed  for  return  of  investment  for  a reasonable  period  of  time. 

4.  Deliverables 

a)  Written  report  describing  all  the  project  activities  as  described  within  this  document. 

5.  Rubric 


Attributes 

C (79-70) 

B(89-80) 

A (100-90) 

Marks 

Concept  designs 
and 

understanding  of 
the  problem 

Demonstrate  partial  understanding 
of  the  problem: 

Only  two  designs 
Unclear  design  descriptions 
- Improper  use  of  technical 
terms 

Demonstrate  understanding  of  the 
problem: 

- All  three  designs 

- Provided  design  descriptions  and 
choose  one  as  the  final  choice 

- Some  use  of  technical  terms 

Demonstrate  complete  and  clear 
understanding  of  the  problem: 

- All  three  designs  with  clear 
description  of  the  designs  with 
proper  diagrams  and  choose  one  as 
the  final  choice. 

- Proper  use  of  technical  terms 

5 

Research  for 
specific 
component 
selection 
(considering  the 
design  criteria) 

Collects  very  little  information: 

- Information  provided  from 
only  two  sources 

- No  review  of  collected 
information 

Collects  some  basic  information,  most 
relates  to  the  problem: 

- Information  provided  from  three 
sources 

- Some  review  of  the  collected 
information 

Collects  great  deal  of  information  with 
proper  analysis: 

- Information  provided  from  three 
different  sources 

- Complete  review  of  the  collected 
information 

25 

Complete  Design 

Basic  design: 

- No  or  very  little  justification 
for  selecting  components 

- System  diagram  with  little 
details 

Moderate  design: 

- Some  justification  for  selecting 
components 

- System  diagram  with  some  details 

Complete  design: 

- Complete  justification  for  selecting 
components 

- System  diagram  with  complete 
details  with  an  orderly  manner 

35 

Evaluation 

Limited  evaluation: 

- Provide  partial  evaluation  of 
the  design  with  some 
description  of  the  evaluation 
process 

- No  suggestion  of  any 
modification 

Full  evaluation: 

- Provide  an  evaluation  of  the  design 
with  some  description  of  the 
evaluation  process 

- Suggest  one  possible  modifications 
in  the  design  in  light  of  the 
evaluation  outcome 

Detailed  evaluation: 

- Provide  a detailed  evaluation 
process  for  all  the  design  outcomes 

- Review  all  the  design  decisions  in 
terms  of  identified  functionalities 

- Suggest  few  possible  modifications 
in  light  of  the  evaluation  outcome 

20 

Written  report 
(A  separate 
document  is 
provided  with  a 
report  sucture) 

Report  with  some  details: 

- Explain  the  project  activities 
using  texts,  and  few  diagrams, 
and  tables. 

- Very  little  use  of  technical 
terminologies 

- Little  use  of  supplied  report 
structure 

- Incorrect  grammar  and 
punctuations 

Report  with  details: 

- Explain  the  project  activities  using 
texts,  diagrams,  and  tables  (as 
necessary) 

- Use  of  technical  terminologies 

- Most  use  of  supplied  report  structure 

- Minor  grammatical  mistakes 

Elaborate  report: 

- Explain  the  project  activities  with 
texts  and  lots  of  diagrams,  and 
tables. 

- Use  of  technical  terminologies 

- Complete  use  of  supplied  report 
structure 

- Good  English  and  punctuations 
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Abstract 

Students  in  an  introductory  engineering  mechanics  (statics)  are  randomly  divided  into  two 
groups.  Both  groups  receive  identical  instruction  except  for  roughly  once  time  per  week,  for 
the  first  half  of  the  semester.  During  these  exceptional  sessions,  one  group  is  given  hands-on 
manipulatives  with  which  to  solidify  concepts,  while  the  other  group  is  not.  The  degree  of 
learning  is  assessed  with  a midterm  multiple  choice  concept  test  and  midterm  problem  solving 
whose  questions  have  multiple  interconnected  parts.  Overall,  the  two  groups  show  no  notable 
difference  in  learning.  However,  when  one  looks  at  electrical  engineering  (EE)  students  and 
mechanical  engineering  (ME)  students  separately,  it  appears  that  the  EE  students  benefit  from 
the  hands-on  exercises,  while  the  ME  students  might  be  better  without. 


1 Introduction 

According  to  current  understanding,  we  humans  think,  learn,  and  solve  problems  by  making  con- 
nections and  associations  to  our  previous  experiences  [3].  It  follows  that  if  one’s  first  exposure  to 
engineering  concepts  takes  place  by  passively  hearing  it  in  a lecture  or  by  reading  it  in  a textbook, 
the  experience  may  not  be  sufficiently  significant  or  rich  to  build  connections. 

Hake  [6]  conducted  a study  of  more  than  6,500  students  in  62  different  introductory  physics 
courses.  He  found  that  students  taking  interactive  engagement  (IE)  courses  had  dramatically  better 
conceptual  understanding,  compared  to  students  taking  traditional  courses.  Here,  Hake  defines 
“interactive  engagement”  (IE)  courses  as 

...  those  designed  at  least  in  part  to  promote  conceptual  understanding  through  inter- 
active engagement  of  students  in  heads-on  (always)  and  hands-on  (usually)  activities 
which  yield  immediate  feedback  through  discussion  with  peers  and/or  instructors. 

In  Hake’s  study,  “traditional”  courses  are  those  that  make  little  use  of  IE  methods.  A partial  list 
of  other  studies  that  corroborate  and  build  upon  Hake’s  findings  include  [9-11, 15,  20-22].  Some  of 
these  other  results  are  even  more  dramatic.  One  particularly  interesting  study  is  that  by  Redish  et 
al.  [13]  who  show  evidence  that  the  gains  derived  from  IE  learning  are  due  to  the  type  of  instruction 
rather  than  differences  in  time  on  task  or  the  skills  of  individual  instructors. 

But  how  critical  is  the  “hands-on”  component  of  interactive  engagement?  In  Section  2.2,  we 
highlight  multiple  viewpoints  expressed  in  the  literature.  Some  researchers  see  hands-on  activities 
as  particularly  effective  techniques  for  developing  conceptual  understanding.  Others,  favoring  a 
more  axiomatic  approach,  see  experimentation  as  a less  important  activity  for  learning.  Herein, 
we  describe  an  experiment  in  which  we  randomly  divided  a large  introductory  mechanics  course 


1 


into  two  groups.  One  group  was  given  objects  that  they  can  manipulate  while  they  they  learn 
and  explore  mechanics  concepts.  The  other  group  learned  via  interactive  engagement,  but  did  not 
have  the  hands-on  manipulatives.  In  this  paper,  we  present  differences  in  learning  between  the  two 
groups.  Some  subsets  of  students  appear  to  benefit  form  the  hands-on  activities  while  others  do 
not. 

The  research  was  conducted  as  part  of  the  College  Initiative  on  Teaching  and  Learning  (CITL) 
within  the  College  of  Engineering  & Engineering  Technology  at  Northern  Illinois  University.  Selected 
faculty  members  from  all  four  departments  in  the  college  participated  in  a series  of  workshops 
where  they  completely  redesigned  courses  they  were  teaching,  posed  educational  research  questions 
associated  with  the  classes,  and  designed  experiments  to  answer  those  questions. 

2 Modeling  in  Engineering  Mechanics 

Before  describing  the  experiment  and  its  results,  it  is  instructive  to  characterize  the  nature  of  the 
engineering  mechanics  course  that  serves  as  our  laboratory. 

Like  most  mechanical  engineering  curricula  throughout  the  country,  we  at  Northern  Illinois  Uni- 
versity (NIU)  require  undergraduates  to  take  an  introductory  mechanics  course  from  the  physics 
department.  Shortly  afterward,  students  must  take  a two  course  sequence  in  “engineering  mechan- 
ics” (statics  and  dynamics),  taught  by  engineering  faculty.  Overlap  between  the  physics  course  and 
the  engineering  courses  is  significant.  Both  the  engineering  and  the  physics  courses  cover  vector 
arithmetic,  Newton’s  laws  of  motion,  impulse-momentum  principles  (linear  and  angular),  and  the 
work-energy  principle.  However,  since  these  topics  are  so  fundamental  to  mechanical  engineering, 
many,  if  not  most,  of  us  find  the  overlap  justifiable. 

In  the  engineering  meehanies  courses,  students  often  have  to  solve  problems  with  multiple  inter- 
connected bodies.  Typically,  this  makes  the  problems  more  difficult  to  solve,  yet  makes  the  problems 
more  relevant  to  engineering.  In  our  opinion,  the  most  important  contribution  that  the  engineering 
mechanics  courses  make  toward  the  undergraduates’  education  is  the  systematic  engineering  ap- 
proach to  modeling,  analysis,  and  problem-solving  that  we  try  to  inculcate  into  the  students  early 
in  the  curriculum.  It  is  the  same  process  that  students  will  use  later  to  study  mechanics  of  materials, 
vibrations  in  mechanical  systems,  the  dynamics  of  fluid  systems,  jet  propulsion,  and  more. 

To  study  the  mechanics  of  machines  and  devices  that  engineers  create,  the  engineer  must  first 
create  a model  of  the  object.  In  engineering  mechanics,  modeling  is  a process  of  abstraction  in 
which  real-world  objects  are  represented  by  mathematical  models  amenable  to  rigorous  analysis. 
Borrowing  from  Hestenes  [7],  we  represent  the  two-step  process  schematically  as  shown  in  Eigure  1. 

The  lowest  level  in  the  figure  represents  the  real  world  itself,  containing  all  the  machines  and 
devices  that  engineers  create.  The  first  step  in  the  modeling  process  that  we  teach  is  to  create  a 
free  body  diagram  (EBD).  The  EBD  is  a graphieal  representation  of  the  physical  object,  depicting 
all  the  forces  and  moments  acting  on  it.  In  creating  the  EBD,  one  usually  has  to  make  assumptions 
or  approximations.  Eor  example,  it  is  common  to  approximate  real  world  objects  as  particles  or  as 
undeformable  bodies.  One  might  choose  to  ignore  the  weight  of  a component,  or  neglect  friction. 
One  may  represent  forces  acting  on  a body  as  concentrated  load  applied  to  a single  point.  The 
forces  that  one  draws  on  the  EBD  must  always  obey  Newton’s  third  law. 

In  the  second  step  in  the  modeling  process,  we  apply  Newton’s  second  law.  Students  translate 
the  graphical  model  into  a mathematieal  model.  The  set  of  equations  that  students  derive  express 
a relationship  between  known  quantities  given  to  them  and  unknown  quantities  that  they  wish  to 
determine.  This  is  “Level  2”  in  Eigure  1. 
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Level  2:Equations  of  motion 


Figure  1:  Schematic  depiction  of  modeling  process  in  engineering  mechanics.  Adapted  from  [7]. 

Assuming  that  one  is  able  solve  the  equations  in  the  mathematical  model  for  the  quantities 
of  interest,  one  realizes  the  powerful  analytical  framework  presented  in  Figure  1.  Solution  of  the 
equations  which  reside  in  Level  2,  corresponds  to  the  “behavior”  of  the  system  depicted  graphically 
in  the  FED  at  Level  1.  If  the  FED  is  a good  representation  of  the  physical  system,  then  the 
mathematical  results  shall  closely  predict  the  actual  behavior  (e.g.  forces,  accelerations)  of  the  real- 
world  system  at  Level  0.  This  process  is  a model  for  how  even  very  complex  engineering  analysis 
works. 

2.1  What  Students  Don’t  Understand  in  Statics 

Statics  is  the  first  of  the  two  engineering  mechanics  courses  we  teach.  In  this  course,  none  of  the 
systems  accelerate.  Setting  a = 0 in  Newton’s  second  law,  converts  the  equations  of  motion  in  to 
form  of  ordinary  differential  equations  (ODEs)  into  equilibrium  equations  in  the  form  of  algebraic 
equations.  Often  the  algebraic  equations  are  linear.  As  a consequence,  they  are  relatively  easy  to 
solve.  Thus  when  students  have  difficulties,  the  difficulties  lie  in  creating  an  appropriate  free  body 
diagram  or  in  deriving  the  equilibrium  equations  - levels  1 and  2 of  Figure  1. 

The  assertion  is  backed  up  by  a recent  paper  by  Streveler  et  al.  [19].  They  report  initial  findings 
of  a Delphi  study  on  engineering  mechanics  concepts  that  students  find  most  difficult.  The  most 
difficult  statics  concepts  on  the  list  are:  1.  static  indeterminacy;  2.  external  vs.  internal  forces; 
3.  isolating  a body  from  its  surroundings;  4.  couples;  5.  static  friction;  6.  importance  of  signs  on 
forces;  7.  distributed  forces;  and  8.  two  force  members.  According  to  the  study,  item  1 is  the  most 
difficult  among  concepts  in  the  list,  while  items  2 through  7 all  tie  for  next  most  difficult.  All  these 
difficult  concepts  lie  within  the  modeling  phase  of  the  problem  solving  process.  (Although  items  1 
and  5 may  also  reside  in  the  solution  phase.)  Interesting,  but  perhaps  not  surprising  to  those  who 
teach  statics,  is  that  most  of  the  difficult  concepts  are  related  to  drawing  appropriate  free  body 
diagrams. 

Also,  it  is  worth  noting  that  the  difficult  concepts  are  not  natural  concepts  that  the  average 
person  recognizes  as  he/she  casually  observes  the  physical  world.  To  internalize  the  concepts, 
students  must  engage  in  deep  thought,  possibly  supplemented  by  experimentation.  Furthermore, 
students  must  recognize  and  abandon  any  misconceptions  they  have. 
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2.2  Perspectives  on  Interactive  Engagement 

Most  engineering  instruction  is  deductive.  Theories  and  general  principles  are  taught  first;  appli- 
cations then  follow.  The  practice  appears  to  support  a widely  held  misconception  in  engineering 
education  practice  that  knowledges  is  something  that  can  be  simply  transmitted  from  expert  to 
novice. 

Decades  of  research,  however,  supports  an  alternative  model  of  learning  (constructivism)  in 
which  knowledge  is  constructed  rather  than  absorbed.  Within  this  framework,  knowledge  is  gained 
only  after  the  new  information  filters  through  a student’s  mental  structures  that  “incorporate  the 
student’s  prior  knowledge,  beliefs,  preconceptions,  misconceptions,  prejudices,  and  fears.”  [12]  New 
information  that  is  consistent  with  the  existing  mental  structures  is  more  likely  to  be  integrated. 
Contradictory  information,  or  information  that  simply  does  not  connect  with  the  existing  mental 
structures,  more  often  passes  through  without  being  incorporated  into  the  knowledge  base. 

From  this  perspective.  Prince  and  Felder  [12]  argue  that  engineering  instruction  should  be  in- 
ductive. Educators  should  start  with  applications  that  provide  meaning  and  context  to  students; 
then  we  can  build  the  theory  on  top  of  applications  where  the  questions  “why?”  and  “how?”  can  be 
answered  readily.  Prince  and  Felder  define  the  term  inquiry  learning  as  instruction  that  uses  ques- 
tions and  problems  to  provide  context  for  learning,  and  which  does  not  fall  into  related  but  more 
restrictive  categories  such  as  problem-based  learning,  project-based  learning,  case-based  learning, 
discovery  learning,  and  just-in-time  teaching  [12]. 

2.2.1  Instruction  Approach  #1:  Hands-On,  Heads- On.  Laws  et  al.  [9]  provide  a some- 
what narrower  working  definition  of  inquiry-based  learning  in  their  studies  of  student  learning  in 
physics.  Primary  elements  of  their  instructional  approach  are  listed  in  Table  1. 


1.  Use  peer  instruction  and  collaborative  work. 

2.  Use  activity-based  guided  inquiry  curricular  materials. 

3.  Use  a learning  cycle  beginning  with  predictions. 

4.  Emphasize  conceptual  understanding. 

5i.  Let  the  physical  world  be  the  authority. 

6.  Evaluate  student  understanding. 


Table  1:  Elements  of  inquiry  based  learning.  Adapted  from  [9]. 


We  call  specific  attention  to  item  5i  of  the  list.  In  this  instructional  model,  the  instructor  is  not 
the  authority.  Neither  is  the  textbook,  nor  the  answers  in  the  back  of  the  textbook.  Instead,  students 
have  access  to  a physical  system  that  they  can  probe,  test  hypotheses,  and  verify  understanding 
of  particular  questions  they  are  asked.  In  this  instruction  model,  students’  small-scale  experiments 
play  a key  role  in  concept  acquisition. 

Steif  and  Dollar  [18]  and  Thacker  et  al.  [20]  espouse  a similar  approach  of  “hands-on,  heads-on” 
learning.  All  these  studies  yield  rather  dramatic  improvements  in  learning  compared  to  students 
who  take  traditional  lecture-based  courses. 

In  a similar  approach,  Thornton  and  Sololoff  [21]  when  digital  simulations  are  used  as  a substitute 
for  the  real  world.  Learning  results  are  similarly  impressive. 

2.2.2  Instruction  Approach  #2:  Hands-Off,  Heads-on  As  an  alternative  to  hands-on 
approach  described  in  the  previous  section,  we  also  consider  a second  instruction  model  motivated 
by  a perspective  proffered  by  Hestenes  [7].  He  argues  that  the  best  way  to  teach  mechanics,  and 
physics  in  general,  is  to  make  a sharp  distinction  between  the  (real)  “Physical  World”  and  the 
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(conceptual)  “Newtonian  World.”  The  Newtonian  World  (upper  two  levels  of  Figure  1)  is  where 
mechanics/physics  lives.  It  is  defined  by  a set  of  axioms:  the  axioms  of  geometry  and  Newton’s 
laws  of  motion. 

At  first  glance,  Hestenes’s  approach  may  appear  rigidly  deductive.  The  axioms  or  laws  are 
presented  upfront;  then  it  is  up  to  students  to  apply  them  to  a litany  of  problems.  However, 
Hestenes  compares  his  approach  to  that  of  playing  a game.  Like  chess,  Hestenes’s  Newtonian 
modeling  game  has  relatively  few  clearly  defined  rules  which  can  lead  to  a rich  set  of  outcomes. 
Unlike  chess,  the  modeling  game  is  not  competition  between  players.  Rather,  it  is  more  like  a puzzle 
in  which  students  are  challenged  to  create  representations  of  the  physical  world  that  are  consistent 
with  the  Newtonian  axioms.  Students  are  encouraged  to  consider  multiple  representations,  to  test 
them,  and  to  recognize  patterns.  In  this  instruction  approach,  students  do  not  use  experiments  to 
acquire  physical  concepts.  Instead,  the  role  of  experiments,  if  any,  is  to  validate  Newtonian  models. 

As  Hestenes  describes  [7],  the  instruction  approach  is  well  aligned  with  the  constructivist  model 
of  learning.  In  the  current  study,  we  couch  a form  of  the  Newtonian  modeling  game  in  an  inquiry 
learning  framework,  as  defined  by  Prince  and  Felder  (See  Section  2.2).  Specifically,  we  adopt  the 
instructional  approach  outlined  in  Table  1,  with  one  exception.  In  place  of  item  5i,  we  substitute 
the  following: 

62.  Let  the  axioms  of  mechanics  be  the  authority. 

In  many  ways  this  second  instruction  approach  is  similar  to  those  found  in  typical  textbooks,  but 
placed  in  an  instructional  setting  that  promotes  interactive  engagement. 

3 Research  Question  and  Methods 

Herein,  we  report  on  a research  project  in  which  we  test  the  two  modes  of  teaching  outlined  in 
Sections  2.2.1  and  2.2.2.  To  keep  the  nomenclature  simple,  we  will  refer  to  these  two  instruction 
models  as  the  “hands-on”  and  “hands-off”  approaches.  Specifically,  we  pose  the  following  question: 

• In  an  introductory  engineering  mechanics  (statics)  course,  which  teaching  strategy  is  more 
effective  on  learning  as  reflected  in  midterm  and  final  examinations:  hands-on  or  hands-off. 

In  Section  3.1,  we  provide  a more  detailed  description  of  the  hands-on  and  hands-off  activities. 

Before  the  first  day  of  class,  we  randomly  assigned  students  registered  for  the  course  into  two 
groups.  One  group  would  periodically  receive  the  hands-on  instruction,  while  the  other  received 
hands-off  instruction.  Students  who  registered  for  the  course  late,  were  assigned  to  a relatively 
small  third  group.  Although  the  third  group  received  “hands-off”  instruction,  the  performance  of 
these  students  was  not  included  in  the  statistics  of  the  “hands-off”  group.  Over  the  semester,  a 
handful  of  students  (distributed  roughly  evenly  over  all  groups)  dropped  the  course.  Their  data  are 
not  included  in  the  analysis  also. 

In  the  first  half  of  the  semester,  we  periodically  split  the  class.  The  two  groups  met  in  separate 
classrooms  and  received  different  instruction.  The  split  occurred  about  once  per  week  on  average, 
usually  coinciding  with  the  introduction  of  new  concepts.  On  other  days,  the  entire  class  met  in  the 
same  room.  On  these  days,  we  more  often  focused  on  problem-solving  exercises  similar  to  homework 
assignments  in  common  textbooks  (e.g.  [1,8, 14]). 

The  first  few  times  we  split  the  class,  we  also  split  the  instructors.  In  other  words,  the  teaching 
assistant  and  professor  switched  rooms  half  way  through  the  class  period.  After  a few  attempts,  we 
found  this  to  be  too  disruptive  to  the  flow  of  the  class.  For  the  remainder  of  the  split  sessions,  the 
teaching  assistant  and  professor  alternated  between  groups  on  separate  days. 
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During  the  split  sessions,  and  also  during  many  of  the  combined  sessions,  students  worked 
in  small  groups  of  three  or  four  students.  The  small  groups  were  assigned  at  the  beginning  of 
the  semester  with  the  goal  of  achieving  diversity  in  academic  major,  and  number  of  years  at  the 
university. 

The  periodic  split  sessions  continued  until  we  administered  the  midterm  exams.  Afterward, 
both  groups  received  common  instruction.  There  was  a hands-on  project  in  which  all  students 
participated  in  designing  and  building  trusses.  Also  the  instructor,  rather  than  students,  performed 
demonstrations  in  the  second  half  of  the  course. 

3.1  Student  Activities 

3.1.1  A pulley  problem.  The  differences  between  the  two  teaching  approaches  may  be  illus- 
trated through  an  example.  At  the  end  of  the  first  week  of  class,  we  ask  students  to  analyze  the 
system  shown  in  Figure  2. 


Both  groups  of  students  are  asked  to  examine  how  the  tensions  in  String  L (T^),  the  upper  part 
of  String  R and  the  lower  part  of  String  R (Tr^)  change  as  one  varies  the  angle  cj)-  Also, 

students  are  asked  to  investigate  what  happens  to  angle  6 as  one  changes  4>. 

The  investigation  should  be  described  as  a preliminary  study,  aimed  at  getting  students  engaged 
in  a problem  before  the  standard  textbook  analysis  is  performed  in  class.  At  the  time  students 
receive  the  assignment,  we  had  covered  Newton’s  laws  in  lecture,  and  the  meaning  of  “force.” 
Furthermore,  we  had  completed  an  in-class  exercise  in  which  students  studied  the  nature  of  tension 
in  a straight  rope  without  a pulley.  Students  found  that  tension  is  essentially  constant  along  the 
length  of  a light-weight  string.  Tension  varies  linearly  along  heavy,  hanging  chains. 

Both  groups  begin  the  exercise  by  hypothesizing.  Students  sketch  plots  of  Tl,  Tr^,  Tr^,  and  0, 
separately,  as  functions  of  (/>.  Based  on  previous  activities,  students  were  generally  able  to  recognize 
that  Tr^  should  equal  the  weight  of  the  block,  regardless  of  the  angle  (/>.  Students  use  their  intuition 
or  prior  knowledge  to  form  the  other  hypotheses. 

Next,  they  explore.  We  give  students  in  the  “hands-on”  group  string,  springs  (for  measuring 
tension),  a pulley,  protractor,  ruler,  and  block  to  carry  out  the  physical  experiment.  Plotting  the 
tensions  and  angle  6 as  functions  of  students  observe  trends.  Specifically,  they  discover  that 
does  not  change  with  (/>;  it  is  always  equal  to  the  weight  of  the  block. 
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The  “hands-ofF”  group  does  not  get  hardware  to  manipulate.  Instead  they  are  asked  to  consider 
the  forces  acting  on  the  pulley  (assuming  one  can  neglect  the  weight  of  the  pulley).  Students  are 
asked  to  depict  the  three  forces  on  the  pulley  graphically  as  indicated  Figure  3a.  Magnitudes  and 


Figure  3:  Vector  addition  of  external  forces,  demonstrating  equilibrium. 

directions  of  the  forces  are  represented  by  lengths  and  directions  of  the  arrows.  In  order  for  the 
pulley  to  remain  in  equilibrium,  all  the  forces  must  balance  out.  Therefore,  when  arranged  head- 
to-tail  the  three  arrows  close  to  form  a triangle.  Students  were  told  to  assume  that  the  tensions 
in  the  upper  and  lower  parts  of  String  R are  identical:  = Tr^.  (Since  this  exercise  occurs 

before  discussion  of  moments,  it  is  not  possible  to  derive  the  result  from  first  principles.)  Armed 
with  a ruler,  protractor,  pencil,  and  paper  they  are  able  to  draw  the  force  triangles  (Figure  3b)  and 
discover  how  the  system  responds  to  changing  cj). 

Before  returning  to  the  next  meeting  of  the  course,  students  in  both  groups  had  to  use  their 
findings  to  compare  and  evaluate  competing  designs  for  a box-hoisting  system.  In  the  next  meeting, 
we  solved  the  problem  in  a more  traditional  manner  in  which  forces  are  decomposed  into  horizontal 
and  vertical  components,  and  equilibrium  equations  are  formulated  and  solved. 

3.1.2  Geometry  of  Vector  Decomposition  In  the  next  split-class  exercise,  students  were 
asked  to  consider  a 200-pound  person  standing  on  a scale.  When  the  scale  lies  flat  on  the  floor,  it 
reads  200  pounds.  What  does  the  scale  read  when  it  lies  on  an  inclined  surface?  What  is  the  scale 
reading  as  a function  of  angle  4>  in  Figure  4? 


Figure  4:  Bathroom  scale  used  in  a vector  decomposition  activity. 

Again,  we  ask  students  to  form  a hypothesis  based  on  hand  sketches  and/or  intuition.  Then 
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they  get  to  work.  While  students  in  the  “hands-on”  group  work  on  the  problem,  they  have  access  to 
a bathroom  scale,  a board  and  some  blocks  that  they  use  to  check  their  intuition  and  calculations. 
Students  in  the  “hands-off”  group  employ  only  the  analysis  techniques  via  pencil  and  paper  taught 
in  the  course.  To  verify  their  results,  students  in  the  hands-off  group  are  encouraged  to  perform 
sanity  checks  in  which  they  confirm  whether  their  answers  make  sense  when  cj)  = 90°,  0°,  and  —90°. 

3.1.3  Exploring  Moments  About  a week  later,  after  introducing  students  to  moments,  stu- 
dents were  asked  to  consider  a set  of  problems  regarding  the  ‘F’-shaped  frame  shown  at  the  top  of 
Figure  5.  The  frame  lies  in  a horizontal  plane  and  is  pinned  so  that  it  is  free  to  rotate  about  point 


Figure  5:  ‘F’-shaped  frame  for  studying  moments. 

A.  We  asked  the  students  nine  questions,  three  of  which  are  shown  in  the  bottom  half  of  Figure  5. 
Under  the  loading  conditions  shown,  students  are  told  that  the  system  is  in  equilibrium.  Then 
they  are  asked  which  of  the  following  are  true:  (A.)  0 < iFil  < IF2I;  (B.)  0 < |Fi|  < \F2\;  (C.) 
0 < \Fi\  = |F'2|;  or  (D.)  either  iF'il  or  |F'2|  must  be  zero. 

Students  in  the  “hands-on”  group  received  physical  representations  of  the  ‘F’-shaped  frames  as 
depicted  in  the  bottom  right  corner  of  Figure  reffig:Fframe.  They  can  apply  the  forces  by  attaching 
springs  to  the  frame  and  pulling.  One  can  feel  which  force  is  bigger  and  see  which  is  bigger  by 
observing  the  relative  lengths  of  the  stretched  springs. 

Again,  the  “hands-off”  group  does  not  get  to  use  the  manipulatives.  Instead,  they  must  solve 
the  problems  by  intelligent  decomposition  of  position  and  force  vectors  and  by  the  principle  of 
transmissibility. 

3.1.4  Tipping  criterion  for  crane.  One  of  the  goals  of  elementary  mechanics  courses  is  to 
develop  students’  problem  solving  skills.  Consider  the  crane  shown  in  Figure  6a.  At  some  point  in 
the  course,  students  become  relatively  proficient  at  performing  straightforward  force  calculations. 
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For  a given  crane  load,  for  example,  one  can  calculate  the  support  forces  that  the  ground  exerts  on 
the  tracks  of  the  crane. 


(b) 

i: 

V 

Figure  6:  Tipping  crane  activity. 

One  can  exercise  students’  higher  order  thinking  skills,  by  rephrasing  the  problem.  For  example, 
we  asked  students  to  calculate  the  minimum  load  that  will  cause  the  crane  to  tip  over.  Novices 
are  often  confounded  by  such  problems.  They  often  attempt  to  incorporate  the  dynamics  of  the 
crane  tipping  into  their  formulation.  Of  course,  the  proper  approach  is  to  formulate  it  as  a statics 
problem,  and  then  calculate  the  load  for  which  the  assumption  of  it  being  a statics  problem  is  on 
the  verge  of  being  violated.  In  this  problem  the  violation  occurs  when  the  normal  force  on  the 
cranes  track  at  B becomes  zero. 

When  solving  the  problem,  students  in  the  “hands-on”  group  get  a toy  crane  made  from 
Legos"’"'^to  play  with.  If  or  when  a small  group  of  students  got  stuck,  the  instructor  guided  them 
through  a simple  hands-on  experiment.  One  student  lets  the  crane  rest  in  his  or  her  hands  as  shown 
in  Figure  6b.  One  track  rests  in  the  student’s  right  hand,  while  the  other  track  rests  in  the  left. 
Without  any  load,  the  crane’s  weight  is  distributed  roughly  evenly  between  the  two  hands. 

When  another  student  pulls  down  on  the  hook,  slowly  increasing  the  load,  the  student  holding 
crane  feels  the  force  one  hand  increase  while  the  force  on  the  other  hand  decreases.  The  student 
observes  that  the  normal  force  vanishes  as  one  of  the  tracks  lifts  off  his  or  her  hand,  and  the  crane 
begins  to  tip.  The  tipping  criterion  to  put  in  the  mathematical  formulation  becomes  clear. 

Students  in  the  “hands-off”  group  do  not  get  cranes.  When  they  run  into  trouble,  the  instructor 
guides  small  groups  of  students  through  a similar  thought  experiment  in  which  they  examine  the 
solution  of  the  static  equilibrium  equations  as  the  load  is  slowly  increased. 
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3.1.5  Design  of  a crane  boom.  In  another  in-class/split-session  activity,  we  asked  students 
to  evaluate  four  crane  designs,  two  of  which  are  shown  in  Figure  7.  For  the  two  cranes  shown  in  the 
figure,  the  one  has  the  secondary  boom,  while  the  other  does  not.  To  figure  out  the  purpose  that  the 
secondary  boom  serves,  students  are  asked  to  investigate  cable  tensions  in  the  two  configurations. 
Again,  students  in  the  “hands-on”  group  are  able  to  tinker  with  the  Lego  cranes  while  those  in  the 
“hands-off”  group  can  not. 


Figure  7:  Different  crane  designs  for  students  to  evaluate. 


3.1.6  A friction  problem.  One  of  the  teaching  objectives  of  the  course  is  to  introduce  students 
to  the  static  Coulomb  friction  model.  In  a class  following  an  introduction  to  static  friction,  students 
were  asked  to  consider  the  two  problems  depicted  in  Figure  8.  In  both  problems,  a cart-like  device 
rests  on  an  inclined  plane.  The  cart  has  one  set  of  wheels  that  are  free  to  rotate;  it  has  a skid  which 
provides  friction  to  keep  the  cart  in  place.  In  one  problem,  the  cart  is  oriented  so  that  the  skid  is 
downhill  of  the  wheels.  In  the  other,  the  skid  is  uphill.  Students  were  asked  to  find  the  maximum 
angle  (j)  for  which  cart  will  remain  stationary  on  the  surface.  Many  novices  believe  that  the  angle 
should  be  the  same  in  both  cases.  After  all,  it  is  the  same  cart. 

Students  in  the  “hands-on”  group  are  given  a toy  cart  made  from  Legos"*"'^and  ancillary  equip- 
ment to  try  out  the  two  case.  Perhaps  contrary  to  their  preconceptions,  students  see  that  the 
configuration  in  8a  is  able  to  remain  stationary  on  a steeper  slope.  Then  they  perform  analysis  to 
explain  the  phenomenon.  Again,  students  in  the  “hands-off”  group  must  solve  the  problem  starting 
from  axioms. 
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Figure  8:  A static  friction  problem. 
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3.1.7  Static  indeterminacy.  In  one  of  the  split  sessions,  we  ask  students  to  find  the  three 
string  tensions  in  the  planar  problem  shown  in  Figure  9a.  It  looks  similar  to  a problem  we  studied  at 
the  beginning  of  the  semester,  and  they  dive  in.  They  are  able  to  derive  two  equations  which  reflect 
horizontal  and  vertical  force  balances.  They  are  unable  to  generate  a third  independent  equation 
by  taking  a moment  about  any  point.  Thus,  they  are  unable  to  solve  for  the  three  unknowns. 


Figure  9:  A statically  indeterminate  problem. 

The  problem  is  an  example  of  static  indeterminacy,  and  it  is  confusing  to  students.  What 
seems  like  a well-posed  problem  has  no  solution;  it  should  be  possible  to  calculate  tensions.  We 
give  students  in  the  “hands-on”  group  materials  with  which  they  can  create  the  three-string  planar 
system.  They  quickly  discover,  that  it  is  not  easy  to  create.  If  string  2 is  a couple  millimeters  too 
short,  then  string  3 goes  slack  (Figure  9b).  Likewise,  if  string  3 is  a couple  millimeters  too  short, 
string  2 goes  slack  (Figure  9c).  Therefore,  it  is  not  a well-posed  problem.  The  problem  becomes 
well-posed,  however,  when  one  replaces  one  or  more  of  the  strings  by  springs  that  stretch.  The 
tensions  depend  on  the  relative  stiffnesses  of  the  springs. 

To  help  the  “hands-off”  students  wrap  their  heads  around  the  conundrum,  we  guided  them 
through  a thought  experiment.  It  is  clear  that  string  1 is  not  necessary  for  equilibrium  when 
string  2 is  in  place,  and  vice  versa.  Therefore,  we  asked  what  happens  when  we  replace  one  of 
these  strings  by  a wet  spaghetti  noodle.  Clearly  the  noodle  cannot  support  more  than  a minute 
fraction  of  the  10-lb  weight.  The  answer  to  the  problem  depends  on  properties  of  the  “strings”.  It 
is  information  not  included  in  standard  statics  analyses. 

3.1.8  Two-force  bodies.  In  another  activity,  we  ask  students  to  find  the  support  force  at 
pin  A for  the  system  shown  in  Figure  10a.  The  activity  occurred  a little  more  than  a week  before 
we  began  talking  about  trusses  and  frames  in  earnest.  To  solve  the  problem,  we  told  students  to 
begin  by  drawing  a free  body  diagram  of  the  part  AC.  After  drawing  a FBD  similar  to  that  shown  in 
Figure  10b,  students  become  stumped.  There  are  four  unknowns  in  the  pin  forces  at  A and  B.  Yet, 
there  are  only  three  independent  equations  one  can  derive.  It  looks  like  a statically  indeterminate 
system.  Nonetheless,  students  are  asked  to  take  a closer  look  at  part  DB. 

Those  in  the  “hands-on”  group  get  long  slender  rods  like  those  shown  in  Figure  10c.  Students 
are  able  to  grasp  the  rod  at  two  pin  joints.  By  handling  and  manipulating  the  rods,  they  discover 
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Figure  10:  A problem  illustrating  the  characteristics  of  a two-force  body. 
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that  forces  in  the  two-force  body  must  be  collinear. 

Students  in  the  hands-off  group  are  guided  toward  this  result  in  the  traditional  axiomatic  way: 
by  drawing  a free  body  diagram  and  setting  up  equilibrium  conditions. 

Armed  with  is  the  collinearity  result  students  in  both  groups  are  able  to  eliminate  one  of  the 
unknowns  in  the  problem  and  solve  for  the  reaction  forces.  Finally,  we  discuss  on-shaped  two-force 
bodies  like  that  shown  in  Figure  lOd. 

4 Results 

To  test  the  effectiveness  of  the  teaching  approaches,  we  collected  five  measures  of  student  learning. 
The  first  two  occurred  near  the  half-way  point  in  the  semester  when  we  administered  two  midterm 
exams.  The  first  exam  was  a multiple  choice  concept  test  that  students  took  on  a Friday.  Immedi- 
ately after  the  exam,  solutions  were  posted  online  so  that  students  could  receive  feedback  on  which 
concepts  were  not  clear. 

The  following  Monday,  students  took  a problem  solving  test  consisting  of  two  traditional  me- 
chanics exercises,  similar  to  the  homework  problems.  On  the  problem  solving  test,  students  were 
asked  to  demonstrate  that  that  they  are  able  to  complete  the  entire  problem  solving  process  from 
drawing  correct  free  body  diagrams;  resolving  forces  into  appropriate  components;  constructing 
equilibrium  equations;  solving  the  equations;  and  making  a correct  interpretation  of  the  result.  To 
grade  the  problems,  we  used  a rubric  that  assigns  points  to  each  step  of  the  process.  It  is  the  same 
rubric  used  to  grade  homework  problems.  Therefore,  going  into  the  exam,  students  knew  exactly 
what  was  expected  of  them. 

We  conducted  the  final  exam  similarly.  Students  took  a multiple  choice  concept  test  during  the 
second  to  last  lecture  period  of  the  semester.  Solutions  were  posted  immediately  afterward.  Then, 
five  days  later,  students  took  a problem-solving  test  to  complete  the  course. 

The  fifth  assessment  of  learning  was  the  Statics  Concept  Inventory  developed  by  Steif  [17].  It 
was  administered  online  over  the  Internet  during  the  week  preceding  final  concept  test.  Students 
were  not  required  to  take  the  Statics  Concept  Inventory,  and  their  score  was  not  included  in  their 
final  grade.  To  motivate  students  to  take  the  inventory  and  to  take  it  seriously,  we  told  students 
that  it  would  be  good  practice  for  their  final  concept  test. 

According  to  Steif  [16],  the  correlations  between  students’  exam  scores  and  scores  on  several 
categories  of  the  concept  inventory  are  among  the  highest  of  all  schools  participating  in  the  concept 
inventory. 

4.1  Exam  Scores 

In  Figure  11,  we  report  the  overall  results  of  the  five  measures  of  learning.  We  are  particularly 
interested  in  the  differences  between  the  “hands-on”  and  “hands-off”  groups.  Therefore,  in  the 
figure,  we  present  a normalized  difference  between  the  average  scores: 

A^on  Aoff  , , 

5 ■ ^ ’ 

Here,  Aon  and  Aoff  are  the  sample  averages  of  scores  from  “hands-on”  and  “hands-off”  groups.  Also, 

5 is  the  standard  deviation  of  the  relevant  sample.  The  error  bars  represent  the  95%  confidence 
intervals  for  each  of  the  measures,  as  defined  by  the  corresponding  t-distributions.  The  error  bars 
are  symmetric,  so  each  truncated  tail  represents  2.5%  probability. 
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Figure  11:  Five  measures  of  student  learning  for  the  entire  class,  just  the  mechanical  engineering 
students,  and  just  the  electrical  engineering  students.  Scores  are  given  by  Equation  (1).  The  learning 
measures  are:  (1)  midterm  concept  test,  (2)  midterm  problem-solving  test,  (3)  final  concept  test, 
(4)  final  problem  solving  test,  (5)  the  Statics  Concept  Inventory. 


4.1.1  Consideration  of  the  class  as  a whole.  Figure  11  shows  that  when  one  looks  at  the 
class  as  a whole,  there  is  little  if  any  discernible  difference  between  the  hands-on  and  hands-off 
groups.  Upon  performing  standard  two-tailed  t-tests  for  difference  of  means,  we  obtain  p = 0.942, 
0.176,  0.616,  0.503,  and  0.469  for  the  five  measures  respectively.  At  first  glance,  it  is  difficult  to  say 
that  that  either  group  performed  better  as  a consequence  of  their  particular  learning  experiences. 

4.1.2  Consideration  of  mechanical  and  electrical  engineering  students  separately. 
The  picture  becomes  potentially  more  interesting,  however,  when  we  consider  mechanical  engi- 
neering students  and  electrical  engineering  students  separately.  On  average,  mechanical  engineering 
students  in  the  “hands-off”  group  performed  better  than  than  their  counterparts  in  the  “hands-on” 
group  in  all  five  measures.  However,  we  cannot  assert  with  95%  confidence  that  the  difference  is 
statistically  significant  since  the  two-tailed  t-tests  yield  p = 0.307,  0.086,  0.136,  0.777,  0.845. 

When  we  consider  the  performance  of  electrical  engineering  students,  we  do  see  a statistically 
significant  difference  in  the  midterm  concept  test.  On  average,  electrical  engineering  students  in 
the  “hands-on”  group  performed  almost  one  standard  deviation  better  on  the  midterm  concept 
test  than  their  counterparts  in  the  “hands-off”  group.  The  two-tailed  t-test  on  this  statistic  yields 
p = 0.037,  indicating  statistical  significance.  Electrical  engineering  students  in  the  “hands-on” 
group  performed  better  on  all  four  other  learning  measures  as  well.  The  average  differences  were  as 
large  or  larger  than  all  other  average  differences  observed.  However,  the  error  bars  for  the  electrical 
engineering  students  were  particularly  large  due  to  the  relatively  few  electrical  engineering  students 
in  the  course.  There  were  18  electrical  engineering  students  (8  hands-on,  10  hands-off)  compared 
to  65  students  (31  hands-on,  34  hands-off)  overall.  Had  the  sample  size  been  larger,  the  differences 
in  the  other  measures  may  have  been  statistically  significant  as  well. 

4.2  Test  Details 

Since  splitting  the  class  into  two  groups  only  persisted  for  the  first  half  of  the  semester,  the  most 
likely  place  to  see  the  different  effects  of  the  teaching  strategies  is  in  the  midterm  exams.  This 
is  corroborated,  at  least  in  part,  by  the  summary  data  in  Figure  11.  In  Figure  12,  we  present 
more  details  of  the  two  midterm  exams.  Data  in  Figure  12  come  in  pairs,  showing  how  mechanical 
and  electrical  engineering  students  separately  performed  in  each  of  the  categories.  As  before,  the 
data  indicate  a difference  in  average  performance  between  the  “hands-on”  and  “hands-off”  groups. 
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Figure  12:  Categorical  details  of  the  midterm  problem-solving  and  concept  tests.  Two  sets  of  data 
are  shown:  one  for  electrical  engineering  students  (solid  error  bars)  and  the  other  for  mechanical 
engineering  students  (dashed  error  bars)  in  the  course.  Scores  are  normalize  according  to  Equation 
(1).  Specific  categories  are  outlined  in  the  text. 


normalized  by  the  standard  deviation  (Equation  1). 

The  first  two  pairs  labeled  “PSl”  and  “PS2”  indicate  difference  in  performance  on  the  two 
questions  on  the  problem-solving  exam.  The  first  problem  (PSl)  is  one  somewhat  similar  to  the 
scale  problem  discussed  in  Section  3.1.2.  By  choosing  to  decompose  the  forces  acting  on  an  object 
into  directions  tangent  and  perpendicular  to  a cable,  the  equations  of  static  equilibrium  become 
especially  easy  to  solve.  Otherwise  it  takes  a bit  more  work.  Problem  (PS2)  is  one  for  which 
students  need  to  determine  the  conditions  under  which  an  object  is  on  the  verge  of  tipping,  similar 
to  the  crane  discussed  in  Section  3.1.4. 

The  remaining  nine  pairs  of  data  are  categories  in  the  midterm  concept  test. 

• Category  VDC.  This  is  a series  of  straightforward  vector  decomposition  problems  in  which 
students  were  asked  to  express  vectors  into  mutually  perpendicular  but  nontrivial  basis  vectors. 

• Category  PUL.  This  is  a set  of  pulley  problems  in  which  students  had  to  recognize  that  tension 
on  the  two  sides  of  a cord  laced  through  a frictionless  pulley  are  equal. 

• Category  GEO.  Problems  within  this  category  test  whether  students  can  recognize,  based  on 
the  geometry  of  the  problem,  whether  some  forces  are  bigger  than  other  forces.  The  scale  in 
Section  3.1.2  is  an  example  of  such  a problem. 

• Category  ERC.  This  is  a sequence  of  problems  that  test  students’  understanding  and  miscon- 
ceptions about  the  standard  Coulomb  model  of  static  friction. 

• Category  MOM.  This  is  a set  of  straightforward  but  nontrivial  problems  similar  to  those 
described  in  Section  3.1.3,  which  assess  students  ability  to  calculate  and  evaluate  relative 
magnitudes  of  moments. 

• Category  EBD.  This  is  a series  of  problems  that  test  whether  students  are  able  to  select  correct 
free  body  diagrams  of  systems  with  pins,  slots,  rollers,  and  systems  with  negligible  friction. 

• Category  2EC.  Problems  that  test  whether  students  are  able  to  recognize  when  bodies  are 
two-force  bodies  and  select  a free  body  diagram  that  properly  reflects  the  configuration. 
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• Category  MAP.  This  is  a set  of  moment  application  problems.  To  receive  points  in  this 
category,  students  must  recognize  the  relative  magnitudes  of  moments  and  determine  whether, 
as  a consequence,  certain  forces  are  larger  than  other  forces. 

• Category  EQM.  A set  of  squares  are  shown  with  forces  acting  on  them.  Students  are  asked 
whether  it  is  possible  for  the  squares  to  be  in  equilibrium. 

As  before,  the  data  in  Figure  12  expresses  the  difference  between  average  scores  in  each  of  the 
categories,  normalized  by  standard  deviation  as  expressed  in  Equation  (1). 

Many  of  the  problems  in  the  midterm  concept  test  do  not  fit  uniquely  into  the  categories  listed 
in  Figure  12.  The  two-force  problems  (2FC)  problems,  for  example,  fit  naturally  as  a subset  free 
body  diagram  (FED)  problems.  Therefore,  for  some  problems,  a correct  answer  would  yield  credit 
in  two  categories  presented  in  Figure  12.  (In  calculating  students’  grades,  they  did  not  get  multiple 
credit.)  Because  we  have  problems  that  span  multiple  categories,  interpretation  is  not  as  clean  as 
one  would  prefer  in  a research  project. 

4-2.1  Electrical  engineering  (EE)  students.  In  the  free  body  diagram  (FED)  category,  we 
see  that  EE  students  in  the  “hands-on”  group  score  significantly  higher  (p  = 0.022,  two-tailed)  than 
their  counterparts  in  the  “hands-off”  group  in  the  FED  (free  body  diagram)  category.  As  discussed 
in  Section  2.1,  this  is  one  of  the  most  important  concepts  for  students  to  learn. 

Because  there  are  relatively  few  EE  students,  error  bars  on  the  electrical  engineering  score 
difference  are  quite  large.  The  difference  between  the  “hands-on”  and  “hands-off”  EE  students 
needs  to  be  on  the  order  of  one  standard  deviation  in  order  reject  the  null  hypothesis  that  means 
are  the  same  (95%  confidence,  two-tailed).  In  the  FED  category,  the  difference  between  the  averages 
is  1.04  standard  deviations. 

None  of  the  other  categorical  differences  reach  that  large  threshold.  Nonetheless,  electrical 
engineering  students  in  the  “hands-on”  group  consistently  scored  higher  than  those  in  the  “hands- 
off”  group  in  all  other  eight  categories.  Recall  from  Figure  11,  that  the  composite  concept  test 
scores  for  EE  students  were  statistically  significant.  Figure  12  suggests,  and  deeper  investigation  of 
the  raw  data  bears  out  that  the  statistically  significant  difference  in  the  composite  score  is  due  the 
accumulation  of  differences  over  all  categories. 

4-2.2  Mechanical  engineering  (ME)  students.  Although  less  dramatic,  the  story  for  me- 
chanical engineering  students  is  almost  exactly  the  opposite.  Our  ME  students  in  the  “hands-on” 
group  scored  lower  than  students  in  the  “hands-off”  group  in  almost  all  categories.  The  cate- 
gory of  pulley  problems  (PUL)  is  the  only  one  that  reached  the  threshold  of  statistical  significance 
{p  = 0.039).  Interestingly,  the  categories  (PUL,  FED,  2FC,  MAP)  for  which  our  difference  measure 
is  most  negative  for  mechanical  engineering  students  are  the  same  categories  for  which  the  mea- 
sure is  most  positive  for  electrical  engineering  students.  A similar  reflection  property  exists  for  the 
midterm  problem-solving  exam  problems  as  well. 

5 Recap  &:  Discussion 

The  statement  seems  obvious:  students  will  learn  mechanics  concepts  better  when  they  have  the 
opportunity  to  simultaneously  see  and  feel  the  phenomena  through  direct  hands-on  experiments. 
The  study  described  herein,  however,  suggest  that  the  statement  is  not  necessarily  true.  On  the 
whole,  we  found  essentially  no  difference  between  students  who  were  given  hands-on  manipulatives 
and  those  who  were  not.  Both  groups  of  students  received  instruction  that  was  largely  inquiry-based 
and  inductive. 
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Upon  closer  inspection  of  the  data  however,  there  appears  to  be  more  subtle  differences  in  how 
mechanical  engineering  students  and  electrical  engineering  students  responds  to  the  two  teaching 
approaches.  Specifically,  we  found  that  EE  students  in  the  “hands-on”  group  performed  better 
on  a midterm  concept  test  than  their  counterparts  in  the  “hands-off”  group.  The  difference  is 
statistically  significant  in  our  experiment. 

Mechanical  engineering  students  appeared,  in  some  sense,  to  respond  in  the  opposite  way.  Those 
who  were  given  the  hands-on  manipulatives,  on  average,  performed  worse.  In  fact,  they  performed 
worse  on  all  measured  categories  in  the  two  midterm  exams.  Nonetheless,  the  differences  were  not 
large  enough  for  us  to  definitively  rule  out  the  possibility  that  the  result  might  be  due  to  random 
chance. 

We  find  the  different  responses  among  the  EE  and  ME  students  rather  curious  and  unexpected. 
It  is  a result  that  might  lead  one  to  suspect  that  there  are  other  factors  in  play.  Rather  than 
occupy  considerable  space  describing  details  of  non  illuminating  data,  we  simply  mention  that  we 
did  not  find  any  significant  differences  among  the  groups  and  subgroups  in  their  learning  styles 
as  measured  by  Eelder  & Silverman’s  [5]  Index  of  Learning  Styles.  Eurthermore,  there  was  no 
significant  difference  in  students’  mechanical  reasoning  ability  at  the  beginning  of  the  semester 
as  measured  by  the  Differential  Aptitude  Test  for  Personal  Career  Assessment.  The  mechanical 
reasoning  test  measures  the  ability  to  “understand  basic  mechanical  principles  of  machinery  tools 
and  motion.”  “Items  represent  simple  principles  that  involve  reasoning  rather  than  specialized 
knowledge  or  training.”  [2]  The  test  is  designed  for  people  with  no  more  than  a high  school  degree. 

Therefore,  the  differing  response  of  ME  and  EE  students  to  “hands-on”  and  “hands-off”  in- 
struction that  we  observed  here  remains  a mystery.  Anecdotally,  both  the  primary  instructor  and 
the  teaching  assistant  observed  that  students  in  the  “hands-on”  group  might  have  been  having  too 
much  fun.  “Playing”  with  the  Legos"*"^may  have  been  a distraction  from  the  learning  objectives 
of  the  hands-on  activities.  Since  electrical  engineering  students,  almost  by  definition,  are  not  as 
inclined  to  be  as  interested  in  mechanical  gadgets,  perhaps  they  were  better  able  to  focus  on  the 
learning  tasks  and  benefit  from  them.  Nonetheless,  we  do  not  have  data  to  support  or  refute  this 
speculative  conjecture. 

Before  concluding,  we  should  emphasize  that  the  degree  to  which  one  can  generalize  our  findings 
to  other  examples  of  “hands-on”  versus  “hands-off”  instruction  is  unclear.  Certainly,  the  hands-on 
exercises  could  have  been  designed  better.  If  we  had  been  aware  of  Dollar  and  Steif’s  work  on 
creating  hands-on  modules  for  statics  [4],  including  the  way  in  which  they  re-organized  the  class,  at 
the  time  we  began  the  project,  we  likely  would  have  designed  our  hands-on  activities  differently. 

Given  this,  we  hope  the  present  work  provokes  broader  and  deeper  study  into  impacts  that 
hands-on  manipulatives  can  have  on  learning,  and  how  the  impacts  vary  in  different  parts  of  the 
student  population. 
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Abstract 

Traditional  assessments  including  examinations  are  most  common  or 
traditional  form  of  assessment.  At  times  these  are  replaced  or  supplemented  by 
performance  tests  not  only  as  an  alternative  means  of  assessment,  but  also  for 
improvement  of  student  learning  and  performance.  This  paper  presents  a study 
designed  to  examine  three  questions.  The  first  question  was  whether  a 
performance  test  administered  in  conjunction  with  a traditional  cognitive  test 
result  in  increased  learning  beyond  the  traditional  test  alone  as  indicated  by  scores 
on  the  traditional  cognitive  test.  The  second  question  was  whether  the  order  of 
administration  of  a performance  test  and  a traditional  test  result  in  differential 
learning  as  indicated  by  the  combination  of  performance  test  and  traditional  test 
scores.  The  third  question  was  whether  the  order  of  administration  of 
performance  test  and  traditional  test  affect  knowledge  retention  as  indicated  by  a 
final  exam.  Statistical  analysis  of  the  results  is  presented. 

Keywords:  Traditional  tests,  performance  tests,  order  of  administration. 
Effectiveness  of  tests,  knowledge  retention 


Introduction 

Northern  Illinois  University  is  the  second  largest  public  university  in  the 
state  of  Illinois.  It  is  a comprehensive  university  emphasizing  both  teaching  and 
research.  The  college  is  committed  to  build  a regional  and  national  reputation  as 
evidenced  by  its  establishment  of  a program  of  Scholarship  and  Teaching 
Initiative  (Vohra,  2005)  which  has  sponsored  this  study.  The  college  formed  a 
learning  community  (Eewis  and  Allen,  2005)  with  faculty  from  all  four 
departments  of  the  college.  Scarborough  (2007)  conducted  a semester  long 
Eaculty  Development  Program  to  guide  these  faculty  related  to  the  initiative 
including  development  of  Self  Assessment  Baseline,  Course  Analysis,  Student 
Centered  Course  Syllabus,  Multifaceted  Assessment  System,  Traditional 
Objective  Tests,  Performance  Assessment  & Rubrics  etc.  Gilmer  (2007a) 
provided  the  guidance  related  to  development  of  tests  and  assessments  and  their 
statistical  analysis.  One  of  the  operational  objectives  of  this  program  is  studying 
the  effect  of  selected  teaching  strategies  on  student  learning  which  will  also  result 
in  strengthening  teaching  and  student  learning.  This  initiative  is  also 
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complimentary  to  philosophy  about  synergy  of  teaching  and  research  advocated 
by  Boyer  (1990),  Braxton  (1996),  and  Glassick  et  al  (1997). 

Among  various  models  of  assessment  (Marzano  et  al,  1993),  Scarborough 
(2004)  advocated  the  model  of  performance  assessment  for  learning 
(Scarborough,  2004).  In  this  model  performance  assessment  played  a key  role. 
National  Society  for  the  Study  of  Education  also  mentioned  that  performance 
assessments  are  not  arbitrarily  separated  from  learning  and  are  less  artificial 
(Wolf  & Reardon,  1996).  National  Education  Goals  Panel  has  noted  that 
performance  assessments  may  be  more  closely  aligned  with  educational  goals 
than  traditional  tests. 

Performance  assessments  for  the  proposed  study  were  developed 
according  to  the  suggestions  by  the  National  Education  Goals  Panel  so  that  they 
have  following  characteristics: 

1 . Be  open-ended.  Require  the  students  to  construct  a response  or 
perform  an  activity. 

2.  Involve  higher-order  complex  skills.  These  would  include  formulating 
and  solving  problems,  reasoning  and  communication. 

3.  Require  extended  periods  of  time  for  performance.  Include  the 
collection  and  analysis  of  data  as  well  as  preparation  of  written  or  oral 
presentations  of  results  and  conclusions. 

4.  Involve  group  performance. 

5.  Provide  some  latitude  in  the  choice  of  tasks. 

6.  Provide  scoring  guidelines  or  rubrics 

Performance  assessments  along  with  traditional  tests  or  assessments 
provide  the  necessary  balance  (Wiggins,  1998)  in  the  system  required  for 
development  of  students’  ability  to  perform  in  types  of  assessments  where  they 
have  traditionally  shown  weaker  performance.  It  may  be  noted  that  performance 
assessments  can  be  of  different  types  (Chatterji,  2003).  In  our  proposed  study  the 
performance  tasks  may  fall  under  the  categories  of  written  open-ended 
assessments  and  product-based  assessments.  Huba  and  Breed  (2000)  presented 
how  performance  assessments  are  part  of  Eearner-Centered  paradigm  which  are 
more  effective  than  the  Teacher-Centered  paradigm  associated  with  traditional 
assessments.  Performance  assessments  have  also  been  thoroughly  discussed  by 
Nikto  (2004)  where  he  presents  both  advantages  and  disadvantages  of 
performance  assessments. 

It  may  be  noted  that  while  vast  literature  exists  on  performance 
assessments  as  evidenced  by  above  cited  references,  no  study  has  been  made  so 
far  on  the  effect  of  order  of  performance  assessment  and  traditional  assessment 
and  the  proposed  study  is  undertaken  to  fill  that  void. 

In  our  college,  all  undergraduate  Mechanical  Engineering  students  are 
required  to  take  a course  on  Mechanical  Vibration  and  it  was  decided  to  involve 
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this  course  as  part  of  this  Scholarship  and  Teaching  Initiative.  The  course  was 
first  analyzed  to  identify  the  gaps  for  ABET  standards,  Northern  Illinois 
University  general  education,  student  learning  objectives  and  associated  teaching 
models,  teaching  styles,  learning  styles.  Bloom’s  taxonomy  (Bloom,  1956)  and 
Dale’s  Cone  of  Learning  (Dale,  1969)  and  then  student  learning  objectives, 
examinations  (mid  term  and  final),  performance  assessments  and  associated 
rubrics  were  prepared  accordingly.  While  it  is  anticipated  that  active  learning 
(Johnson  et  ah,  1991)  facilitated  by  participating  in  performance  tasks  assigned  in 
this  course  (along  with  appropriate  rubrics,  Nitko,  2004)  will  enhance  student 
learning,  the  order  of  performance  assessments  and  traditional  tests  were  decided 
to  be  explored  as  the  research  question. 

Course  analysis  and  subsequent  changes 

During  the  workshop  for  this  teaching  and  learning  initiative,  the  course 
was  thoroughly  analyzed  in  terms  of  ABET  standards,  NIU  general  education, 
student  learning  objectives,  relation  of  student  learning  objectives  and  teaching 
models,  relation  of  student  learning  objectives  and  teaching  styles,  relation  of 
student  learning  objectives  and  learning  styles,  and  relation  of  student  learning 
objectives  to  Bloom’s  taxonomy  and  Dale’s  cone  of  learning.  Three  performance 
tests  and  associated  rubrics  were  prepared.  Each  test  item  (for  both  mid-term  and 
final)  was  related  to  the  student  learning  objectives.  Detailed  course  calendar  was 
prepared  where  topics  and  associated  teaching  model,  teaching  style,  learning 
style  and  Bloom  and  Dale’s  Cone  of  learning  were  listed  for  each  class  period. 

It  may  be  noted  that  in  the  previous  years  even  though  ABET  outcomes 
were  analyzed  for  the  course;  they  were  not  thoroughly  analyzed  in  terms  of 
student  learning  objectives.  This  course  is  not  a university  defined  general 
education  course  and  as  such  general  education  content  analysis  was  not  relevant 
for  this  course. 

Various  concepts  (Constructivism,  Metacognition,  Scaffolding,  Zone  of  Proximal 
Development,  and  Role  of  Expert  performance)  that  apply  to  learning  were  explored  in 
terms  of  their  application  to  the  course.  Regarding  Constructivism,  even  though  the 
current  course  has  both  direct  and  discovery  methods,  it  was  mostly  direct  method. 
Performance  tasks  brought  in  more  discovery  method  to  create  a balance  of  these  two 
methods.  Regarding  Scaffolding,  initial  assignments  had  more  information  but  later 
assignments  had  less  information.  Regarding  Zone  of  Proximal  Development  (optimal 
mismatches  with  tasks  given  to  students),  it  may  be  noted  that  for  performance  tasks, 
students  initially  worked  individually  and  since  they  are  at  various  stages:  very  rigid  or 
stage  I of  Optimal  Environment  to  adaptable  or  stage  IV  of  Optimal  Environment,  they 
faced  different  challenges.  However,  when  they  worked  together  to  arrive  at  best 
solution,  they  learnt  to  be  adaptable. 

Various  information  processing  models  (Joyce  et  ah,  2004)  for  teaching  were 
studied  during  the  workshop  and  performance  tasks  involved  Scientific  Inquiry  and 
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Advance  Organizers.  Some  problems  in  the  examinations  also  involved  inductive 
thinking.  In  terms  of  social  models  of  teaching,  performance  assessments  relate  to  both 
Structured  Inquiry  and  Group  Investigation.  It  also  involved  Positive  Interdependence. 
Earlier  the  course  was  taught  mostly  as  direct  instruction  (DI)  and  occasional  group 
investigation  (during  laboratory  exercises).  However,  the  revised  course  not  only 
involved  direct  instruction  (DI),  but  also  Structured  Inquiry  (SI).  More  group 
investigation  (GI)  took  place  because  in  addition  to  laboratory  exercises,  performance 
tasks  were  assigned  where  students  start  to  work  individually  and  then  collaborate  as  a 
group. 


Regarding  teaching  styles,  earlier  primarily  command  and  practice  styles 
were  followed.  Now  the  revised  course  also  included  Guided  discovery. 
Reciprocal,  and  convergent  Discovery.  Following  Kolb  (1976),  revised  course 
incorporated  all  the  learning  styles:  Concrete  Experience,  Reflective  Observation, 
Abstract  conceptualization,  and  Active  Experimentation.  Analysis  of  the  course 
in  terms  of  Bloom’s  taxonomy  revealed  that  earlier  the  course  and  associated 
homeworks  and  tests  mostly  involved  the  thinking  levels  of  Knowledge, 
Comprehension,  and  Application.  Rarely  any  assignment  required  students  to 
analyze,  synthesize  and  create  anything.  Revised  course  includes  three 
performance  assessments  all  of  which  required  students  to  perform  at  these  higher 
levels  of  thinking  including  the  highest  level  of  evaluation  and  creation. 

Regarding  Dale’s  Cone  of  learning,  earlier  the  course  format  involved  more 
passive  and  intermediate  learning  and  now  due  to  incorporation  of  performance 
tests,  more  active  learning  took  place. 


Research  Questions  and  Design 

Students  in  this  class  in  fall  2006  were  divided  into  two  groups 
(experimental  group  and  research  group).  Students  were  randomly  assigned  to 
these  groups.  Both  the  groups  were  given  same  instructions,  however  one  of  these 
groups  (experimental  group)  were  assigned  the  performance  test#2  (designing  a 
bicycle  vibration  seat  for  comfortable  ride  from  vibration  point  of  view)  two 
weeks  before  the  traditional  test  (mid  term)  whereas  the  other  group  (control 
group)  was  assigned  the  performance  test#2  after  the  traditional  test  (mid  term). 
Each  group  had  two  weeks  to  work  on  the  performance  test.  It  may  be  noted  that 
both  groups  were  given  the  same  traditional  mid  term  at  the  same  time.  Both 
groups  had  same  traditional  final  exam  and  same  performance  tests  #1  and  #3. 

The  basic  design  for  this  study  is  presented  in  the  table  1 as  shown  below. 
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Table  1.  Performance  Assessment  and  Traditional  Assessment 
Administered  in  Different  Order 


Group  1 
(experimental 
group) 

Instruction 

Performance 

Test#2 

(assigned  Oct  9, 
due  Oct  24) 

Traditional  Test 
(Oct  24) 

Final 

Exam 

Group  2 
control  group) 

Instruction 

Traditional  Test 
(Oct  24) 

Performance 
Test#2  (assigned 
Oct  24,  due  Nov  7) 

Final 

Exam 

The  first  question  of  whether  a performance  test  administered  in 
conjunction  with  a traditional  cognitive  test  result  in  increased  learning  beyond 
the  traditional  test  alone  as  indicated  by  the  traditional  cognitive  test  was 
addressed  by  comparing  the  traditional  test  means  between  the  two  groups. 

The  second  question  of  whether  the  order  of  administration  of  a 
performance  test  and  a traditional  test  result  in  differential  learning  was  addressed 
by  comparing  the  combined  performance  test  and  traditional  test  scores  between 
the  two  groups. 

The  third  question  of  whether  the  order  of  administration  of  performance 
test  and  traditional  test  affect  knowledge  retention  was  addressed  by  comparing 
scores  from  the  final  exam  that  are  based  on  material  also  covered  on  the  midterm 
(the  performance  test  and  traditional  midterm  test)  between  the  two  groups. 


Results 

The  44  students  in  the  class  were  divided  into  two  treatment  groups:  group 
1 comprised  of  22  students  who  took  the  midterm  performance  assessment 
(performance  task  #2)  followed  by  the  traditional  midterm  and  group  2 
comprised  22  students  who  took  the  traditional  midterm  followed  by  the 
performance  assessment. 

The  independent  variable  in  this  context  is  group  assignment.  The  dependent 
variables  depend  on  the  specific  research  questions  addressed.  Statistical  analyses 
of  the  results  were  performed  using  SPSS  [Gilmer,  2007b]  for  each  dependent 
variable  and  they  are  presented  in  the  table  2.  The  statistical  significance  levels 
are  based  on  independent  sample  t-tests. 
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Table  2.  Statistical  Analysis  of  dependent  variables 


Variable 

Group 

N 

Mean 

SD 

Sig. 

Level 

(df=42) 

1 . Traditional  Midterm 

1 . Perf.  First 

2.  Trad,  first 

22 

22 

41.0 

41.2 

8.9 

14.2 

0.97 

2.  Combined  Traditional 
Midterm  & Performance 
Assessment 

1 . Perf.  First 

2.  Trad,  first 

22 

22 

66.8 

69.1 

9.8 

14.4 

0.53 

3.  Final  Exam  - Midterm 
Content  Only 

1 . Perf.  First 

2.  Trad,  first 

22 

22 

25.7 

29.0 

6.4 

7.8 

0.13 

The  analysis  from  table  2 indicates  that  the  difference  between  group  means  for 
variable  1 - the  traditional  midterm  exam,  was  not  statistically  significant  beyond 
the  0.05  level  of  significance. 

Similarly  the  difference  between  the  group  means  for  variable  2 - the  combined 
traditional  midterm  exam  and  the  midterm  performance  assessment,  was  not 
statistically  significant. 

Table  2 also  indicates  that  the  difference  between  the  group  means  for  variable  3 
- the  final  exam  scores  for  the  midterm  content  was  not  statistically  significant. 

Conclusions  and  Recommendations 

Although,  for  all  three  variables,  the  mean  for  students  taking  the  traditional  mid 
term  first  were  higher  than  the  mean  for  students  taking  the  performance  test  first, 
none  of  the  mean  differences  was  statistically  significant.  This  lack  of  statistical 
significance  indicates  that  a null  hypothesis  specifying  equal  effects  of  treatment 
groups  in  this  context  is  still  a viable  hypothesis  and  cannot  be  rejected. 

It  may  be  noted  that  shortly  after  the  semester  started,  there  was  one  performance 
assessment  which  was  given  to  all  students  at  the  beginning  of  the  semester  and 
even  this  performance  assessment  was  not  directly  related  to  the  midterm,  critical 
thinking  ability  of  all  students  increased  possibly  resulting  in  similar  performance 
of  the  two  groups. 

Another  possibility  is  that  students  worked  hard  for  the  first  performance 
assessment  and  might  not  have  spent  enough  time  for  the  second  performance 
assessment  and  in  future  the  first  performance  assessment  may  be  abandoned 
keeping  only  the  second  (related  to  mid  term)  and  the  third  (related  to  final) 
performance  assessment  to  make  a definitive  conclusion. 
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Abstract 

Cooperative  learning  is  defined  as  the  instructional  use  of  small  groups  so  that  students  may  maximize  their  own 
learning  and  each  other’s  learning.  The  benefits  of  cooperative  learning  are  well  known  in  literature.  In  this  paper, 
an  experiment  using  cooperative  learning  is  presented,  which  was  conducted  in  the  first  foundational  Operations 
Research  course  for  the  industrial  and  systems  engineering  program  during  the  fall  semester  of  2006.  The  variable 
“knowledge  retention”  was  used  as  the  performance  measure  in  order  to  study  the  success  of  this  technique.  In 
general,  results  are  promising.  This  paper  presents  the  methodology  used,  the  experiments,  results,  conclusions  and 
future  line  of  research. 

Keywords 

Operations  research,  cooperative  learning,  knowledge  retention,  engineering  education. 

1.  Introduction 

There  is  a clear  and  unmistakable  need  to  improve  teaching  and  learning  methods  in  engineering  education  not  only 
nationwide  [1]  [10-11]  but  also  statewide  - as  asserted  by  Illinois  Commitments’  Goal  2:  “Higher  education  will 
join  elementary  and  secondary  education  to  improve  teaching  and  learning  at  all  levels”  (A  Citizens’  Agenda  for 
Illinois  Higher  Education  -The  Illinois  Commitment;  Partnership,  Opportunities,  and  Excellence;  IBHE,  1999.)  This 
not  only  represents  a call  for  action  but  also  re-activates  what  the  scholarship  of  teaching  in  engineering  education 
should  be  [16]. 

In  a recent  study  that  shows  2004  graduates  as  significantly  better  prepared  than  their  counterparts  from  a decade 
ago,  Lattuca  et  al.  [9]  conclude  the  weight  of  evidence  indicates  that  engineering  education  in  the  US  has  changed 
dramatically  after  the  implementation  of  the  new  ABET  Engineering  Criteria  2000.  These  results  demonstrate  that 
ABET  accreditation  philosophy  based  on  assessments  of  student  learning  and  continuous  improvement  principles 
was  a milestone  in  the  right  direction.  However,  as  pointed  out  by  Sullivan  [14],  “scattered  approaches  to  reforming 
engineering  education  have  not  [yet]  resulted  in  a systemic  change”  and  therefore  call  for  action  to  produce 
substantial  changes  in  engineering  education  remains  far  still  active. 

An  important  aspect  is  the  need  to  prepare  engineering  students  to  function  in  a rapidly  changing  world.  Nationally, 
less  than  55  percent  of  students  who  start  engineering  studies  complete  them.  Kenney  and  Dossani  [7]  say  this  rate 
might  be  severely  affected  during  the  next  years  as  outsourcing  trends  maintain  or  accelerate  in  response  to  ongoing 
pressure  to  reduce  costs  due  to  economic  globalization,  unless  the  engineering  curriculum  adapts  to  address  this  new 
reality.  In  this  new  much  more  globalized  context,  the  prediction  is  that  engineers  of  the  future  might  likely  take  one 
of  two  directions;  either  they  become  global  project  managers  or  entrepreneurs.  Consequently,  Kenney  and  Dossani 
[7]  argue  engineering  students  will  need  an  educational  system  that  provides  them  with  the  tools  to  succeed.  To 
educate  engineers  two  decades  from  now,  the  National  Academy  of  Engineering  (NAE)  suggests  engineering 
courses  should  be  taught  in  such  a manner  that  students  get  engaged  and  their  curiosity  arisen.  In  other  words 
approaches  such  as  active  learning  and  cooperative  (or  collaborative)  learning  should  be  incorporated  in  classrooms 
to  produce  learner-centered,  assessment-centered  and  knowledge-centered  learning  environments.  The  research 
evidence  establishes  that  cooperative  learning  promotes  deep  learning  and  enhances  knowledge  retention  in 
students.  Based  on  this  premise,  this  article  presents  a first  attempt  to  infuse  the  cooperative  learning  approach  into 
the  first  course  of  Operations  Research  (OR),  which  is  part  of  the  undergraduate  core  within  the  Industrial 
Engineering  (IE)  curriculum  at  Northern  Illinois  University.  In  the  following  sections,  a brief  discussion  on 
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cooperative  learning  is  presented  as  well  as  the  methodology  used,  the  experiments,  results,  conclusions  and  future 
line  of  research. 

2.  Cooperative  Learning 

Cooperative  learning  does  not  simply  mean  students  working  in  groups;  it  goes  richly  beyond  and  on  top  of  that. 
Within  the  cooperative  tasks,  individuals  look  for  outcomes  that  are  of  benefit  to  themselves  and  to  all  other  group 
members.  Johnson,  Johnson  and  Smith  [5]  define  cooperative  learning  as  “the  instructional  use  of  small  groups  so 
that  students  work  together  to  maximize  their  own  and  each  other’s  learning  (p.1-14).”  The  disposition  into  groups  is 
to  provide  students  with  an  environment  of  social  interdependence  or  mutual  dependence  rather  than  competition. 
Under  this  idea,  the  way  how  social  interdependence  is  structured  will  determine  how  group  members  interact  which 
in  consequence  affects  the  outcome  achievements.  Cooperation  or  positive  interdependence  results  in  more 
interaction  as  group  members  encourage  and  facilitate  each  other’s  learning.  On  the  contrary,  a negative 
interdependence  or  competition  results  in  oppositional  interaction  as  group  members  obstruct  each  other’s  learning. 
The  absence  of  interdependence  results  in  no  interaction  since  group  members  are  going  to  be  more  focused  on 
individual  efforts  [12].  Extensive  research  has  been  done  in  cooperative  learning  - at  least  600  studies  since  1897 
[12];  most  of  these  studies  show  that  cooperative  learning  favors  higher  individual  achievement  than  both 
competitive  and  individualistic  approaches  [13].  Just  as  an  example.  Hake  [4]  shows  that  interaction  between 
students  during  class  time  is  related  with  a greater  percentage  gain  on  the  Force  Concept  Inventory  (measure  of 
students’  conceptual  understanding  of  mechanics)  when  compared  with  traditional  lecture  courses.  Johnson  et  al.  [5] 
discuss  three  possible  implementations  for  cooperative  learning;  informal  cooperative  learning,  formal  cooperative 
learning,  and  cooperative  base  groups. 

Informal  cooperative  learning  consists  of  having  students  work  together  to  achieve  specific  learning  goals  through 
temporary  groups  that  last  from  a few  minutes  to  one  session  period  [5].  Formal  cooperative  learning  is  subject  to 
the  presence  of  the  following  five  elements:  (1)  positive  interdependence,  team  members  must  be  linked  to  one 
another  in  such  a way  that  they  maximize  their  own  productivity  and  that  of  all  other  group  members;  (2)  individual 
accountability,  all  students  in  a group  are  held  personally  responsible  to  achieve  group  goals  by  doing  their  share  of 
the  work  and  easing  the  work  and  effort  of  other  group  members;  (3)  promotive  (Face-to-Face)  interaction,  team 
members  must  encourage  and  facilitate  each  other’s  efforts  to  complete  tasks  and  challenge  one  another’s 
conclusions  and  reasoning  in  order  to  reach  group  goals;  (4)  social  skills,  students  must  be  taught  and  motivated  to 
develop  and  practice  interpersonal  and  small  group  skills;  and  (5)  group  processing,  team  members  must  reflect  on 
group  member  actions  and  assess  them  to  decide  upon  direction  [5].  Cooperative  base  groups  are  long-term, 
heterogeneous  cooperative  learning  groups  with  stable  membership  where  students’  primary  responsibility  is  to 
provide  one  another  with  encouragement,  to  be  accountable,  and  to  ensure  all  members  are  academically  progressing 

[5]. 

Felder  and  Brent  [3]  affirm  that  a growing  body  of  research  continues  to  confirm  the  effectiveness  of  using 
cooperative  learning  in  higher  education.  Among  the  characteristics  that  cooperatively  taught  students  exhibit  when 
compared  to  students  traditionally  taught  are  the  following:  higher  academic  achievement,  greater  persistence 
through  graduation,  better  high-level  reasoning  and  critical  thinking  skills,  deeper  understanding  of  learned  material, 
lower  levels  of  anxiety  and  stress,  more  positive  and  supportive  relationships  with  peers,  more  positive  attitudes 
toward  subject  areas,  and  higher  self-esteem  [15].  Cooperative  learning  is  not  only  claimed  to  produce  deep 
learning,  which  in  turn  helps  students  to  apply  knowledge  in  other  contexts,  but  also  is  claimed  to  promote  a positive 
attitude  toward  the  subject  matter  and  thus  increase  knowledge  retention  [2]  [6].  When  students  are  successful  their 
self  esteem  enhances  and  they  tend  to  view  the  subject  matter  with  a very  positive  attitude  that  creates  a 
reinforcement  cycle  of  good  performance,  helping  to  improve  both  the  individual’s  and  group’s  self  esteem  [8]. 

The  research  evidence  establishes  that  cooperative  learning  promotes  deep  learning  and  enhances  knowledge 
retention  in  students.  Based  on  this  premise,  this  article  presents  a first  attempt  to  infuse  the  cooperative  learning 
approach  into  the  first  course  of  Operations  Research  (OR),  which  is  part  of  the  undergraduate  core  within  the 
Industrial  and  Systems  Engineering  (ISYE)  curriculum  at  Northern  Illinois  University.  This  study  is  intended  to 
answer  the  two  following  research  questions: 

a)  Does  individual  learning  vi.  cooperative  learning  result  in  differential  knowledge  gains 
as  indicated  by  a traditional  cognitive  test? 

b)  Does  individual  learning  vi.  cooperative  learning  result  in  differential  knowledge 
retention  as  indicated  by  a final  exam? 
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With  respect  to  this  research  matter,  cooperative  learning  has  been  a recurrent  topic  within  the  contents  of  the 
Informs  Annual  Teaching  of  Management  Science  Workshop  during  the  last  three  years.  However,  no  research 
study  addressing  the  effects  of  cooperative  learning  on  OR  knowledge  retention  has  been  found  in  literature  to  date. 

3.  Methodology 

In  order  to  examine  the  impact  of  cooperative  learning  versus  individual  learning  on  knowledge  gain  in  the  form  of 
the  two  research  questions  stated  above,  a sample  consisting  of  18  students  enrolled  in  the  section  of  the  “ISYE  370 
Operations  Research:  Deterministic  Models”  course,  which  is  part  of  the  core  of  the  ISYE  program.  In  this  course, 
students  are  first  exposed  to  fundamental  methods  and  applications  of  deterministic  operations  research  models.  The 
course  was  separated  in  two  overall  groups  and  students  randomly  assigned  to  one  of  them.  Additionally,  working 
teams  were  formed  within  each  overall  group.  Literature  recommends  that  each  team  in  cooperative  learning  should 
be  comprised  by  two  or  three  members  [5].  In  this  experiment,  the  working  teams  were  formed  out  of  two  and  three 
students. 

The  experimental  design  will  consist  of  two  treatments:  (a)  individual  learning  and  (b)  cooperative  learning,  as 
presented  in  Table  1.  After  these  two  treatments  are  performed  there  were  two  posttests  to  collect  data  with  which 
the  research  questions  were  answered. 


Table  1:  Individual  Learning  vs.  Cooperative  Learning 


Course 

Treatment 

Posttest  1 

Posttest  2 

Group  1 

Individual  Learning 

Traditional  Test 

^ 

Pinal 

Group  2 

Cooperative 

Learning 

Traditional  Test 

^ 

Pinal 

The  actual  delivery  of  the  treatment  conditions  alternated  across  content  areas  and  groups,  as  shown  in  Table  2.  Lor 
this  delivery  model  to  work  there  needs  to  be  an  even  number  of  content  areas  with  a minimum  of  two.  Therefore, 
the  following  four  content  areas  were  considered: 

I.  Lormulation/Graphical  Method 

II.  Lormulation  of  Larger  Problem 

III.  Simplex  Method 

IV.  Duality  Theory 


Table  2:  Order  for  Treatments 


Group 

Treatments 

Content  Area 
I 

Content  Area 
II 

Content  Area 
III 

Content  Area 
IV 

1 

Individual 

Cooperative 

Individual 

Cooperative 

2 

Cooperative 

Individual 

Cooperative 

Individual 

Group  1 was  the  individual  learning  group  for  content  areas  I & III  while  Group  2 was  the  individual  learning  group 
for  content  areas  II  & IV.  This  adds  to  the  validity  of  the  design  and  enhances  the  fairness  of  the  treatment 
conditions  within  the  student  groups.  Activities  assigned  during  each  content  area  were  worked  separately  by  both 
groups  and  thus  two  rooms  were  needed  for  this  purpose.  Lor  the  content  areas  where  Groups  1 and  2 became  a 
cooperative  learning  group,  the  group  was  split  into  working  teams  for  the  effectiveness  of  the  approach. 

Lor  fairness  to  students,  each  content  area  was  weighted  approximately  equally  within  both  posttests.  After  the 
administration  of  the  Traditional  Test  (Posttest  1)  the  distinction  between  the  groups  was  dissolved  with  instruction 
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and  assessment  activities  delivered  to  all  students  equally.  Question  1 was  addressed  by  comparing  the  Posttest  1 
means  under  the  individual  and  cooperative  learning  conditions.  Question  2 was  addressed  with  similar  comparisons 
on  the  Posttest  2 means. 

Each  content  area  was  lectured  first  and  then  the  class  was  divided  into  cooperative  and  individual  groups.  Both 
groups  worked  on  the  same  in-class  assignment  during  40  minutes  according  to  certain  instructions.  The  cooperative 
learning  group  was  required  to  follow  the  instructions  and  time  as  shown  in  Figure  1.  Whereas  the  individual 
learning  group  was  basically  required  to  solve  the  problem  in  30  minutes  and  then  students  were  exposed  to  the 
solution  in  10  minutes. 
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Figure  1 ; Instructions  for  cooperative  learning  group 


4.  Results 

In  total  eight  teams  were  formed  and  kept  in  the  class  throughout  the  semester;  six  teams  made  up  of  two  members 
and  other  two  teams  of  three  members.  As  explained  in  the  methodology  section,  after  the  experiment  both  groups 
took  the  midterm  exam.  This  exam  consisted  of  25  multiple-choice  questions  to  make  more  objective  the  evaluation 
of  the  diverse  contents.  Twelve  selected  questions  from  the  midterm  were  again  given  to  the  students  as  part  of  the 
comprehensive  final  exam,  but  the  format  and  the  wording  were  changed.  Table  3 shows  the  results  of  this 
experiment. 


Table  3:  Experiment  results 


Midterm 

Final 

N 

Mean 

StDev 

SE  Mean 

Mean 

StDev 

SE  Mean 

Individual 

18 

29.111 

7.977 

1.880 

3.444 

1.381 

0.325 

Cooperative 

18 

25.778 

9.124 

2.150 

3.333 

1.680 

0.396 

Difference 

18 

3.333 

7.669 

1.807 

0.111 

1.231 

0.290 

95%  Cl  mean  difference 
P-value 

(-0.48069,7.14736) 

0.083 

(-0.501248,  0.723471) 
0.707 

Since  students  within  each  overall  group  alternate  between  individual  and  cooperative  learning  styles  for  different 
contents  as  shown  in  Table  2,  each  student  is  tested  actually  based  upon  both  styles.  Therefore,  a paired-t  test  was 
performed  to  compare  the  difference  between  individual  learning  versus  cooperative  learning.  Table  3 shows  the 
results  of  the  25  and  12  multiple-choice  questions  for  the  Midterm  and  Final  examinations  respectively.  For 
instance,  the  values  of  29.1 1 1 and  25.778  for  Midterm  indicate  the  average  scores  students  obtained  in  questions  for 
those  contents  where  they  worked  under  individual  and  cooperative  learning  styles  respectively.  In  addition.  Table  3 
shows  that  the  95%  confidence  interval  for  the  mean  difference  between  both  types  of  learning  used  does  include 
zero,  which  suggests  there  is  no  statistical  evidence  to  reject  the  null  hypothesis.  The  large  p-values  in  both  cases 
(0.083  and  0.707,  respectively)  further  suggest  that  the  data  are  consistent  with  Hq:  = 0,  that  is,  both  groups 

perform  equally.  This  result  implies  that  for  the  experiment  conducted  both  learning  styles  produced  neither 
differential  knowledge  gains  nor  differential  knowledge  retention  with  respect  to  each  other,  as  response  to  the 
questions  stated  in  Section  2. 


5.  Conclusions  and  Recommendations 
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Results  are  clearly  not  the  ones  expected  according  to  literature  and  thus  they  should  be  interpreted  carefully  and  in 
light  of  this  experiment  only.  In  practice,  it  was  particularly  difficult  to  isolate  within  one  single  course  the  real 
effect  of  using  cooperative  learning  from  individual  learning,  due  mainly  to  the  fact  that  the  students  cannot  be 
treated  unfairly.  Secondly,  a content  size  reduced  makes  difficult  to  treat  the  content  properly,  which  could  have 
happened  in  this  experiment,  but  again  a larger  content  size  may  have  a negative  impact  on  the  fairness.  Third,  the 
sample  size  becomes  an  important  issue  in  this  type  of  study.  Finally,  the  size  of  the  teams  also  had  an  effect  in  the 
results.  Three-member  teams  easily  got  along  and  worked  in  general  better  than  some  of  the  two-member  teams,  in 
which  took  longer  to  unfreeze  members  before  starting  to  work  steadily. 

However,  the  majority  of  the  students  reported  to  feel  highly  pleased  to  work  and  learn  cooperatively  in  teams  on 
their  respective  topics.  It  is  interesting  to  note  that  the  content  related  to  OR  modeling  was  reported  to  be  “hard”  and 
“intimidating”  by  those  who  worked  on  this  topic  individually.  All  students  coincided  on  that  “cooperative  learning 
should  be  kept”  as  a teaching  method  of  all  OR  contents  for  future  courses.  This  is  important  because  one  advantage 
of  working  cooperatively  in  student  teams  - as  instructor  - is  that  teams  were  easier  to  supervise  and  interact  with. 
Instructors  can  observe  students  working  on  assignments  together  and  individually  within  their  teams,  but  when 
students  work  alone  it  is  hard  for  the  instructor  to  observe  the  performance  of  all  of  them.  In  addition,  it  was  noticed 
that  student  participation  in  the  topics  increased,  most  of  them  looked  more  comfortable  with  providing  opinions, 
comments,  and  suggestions  within  their  teams  for  solving  problems  related  to  their  topics. 

The  meta-cognitive  technique  of  repeating  what  students  are  doing  with  each  other  was  reported  to  be  effective  for 
their  learning.  Some  of  the  students  said  they  felt  like  their  ideas  were  reinforced  in  them  while  repeating  and 
explaining  a solution  to  their  partners,  this  is  because  through  this  mechanism  students  were  forced  to  reflect  what 
they  are  learning. 

In  conclusion,  although  results  in  this  experiment  do  not  support  the  evidence  from  experiences  from  other  areas 
reported  in  the  literature  with  respect  to  retention  gains,  neither  are  they  unpromising.  The  infusion  of  this 
cooperative  learning  type  in  the  Operations  Research  course  has  clearly  showed  an  increased  level  of  motivation  and 
participation  in  the  OR  contents.  Therefore,  a recommendation  would  be  to  continue  the  infusion  of  cooperative 
learning  in  OR  topics.  In  the  future,  teams  should  be  formed  not  randomly  as  they  were  in  this  experiment,  but  by 
also  considering  students’  learning  styles.  In  addition,  the  incorporation  of  the  learning  cycle  (Why?  What?  How? 
What  if?)  should  also  be  stressed  in  the  different  activities  of  teams. 
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Instructional  Research  Project 

Learning  at  the  Expert  Level 

“Investigation  of  the  Impact  of  Cooperative  Based  Learning  on  Engineering 

Student  Learning” 

Ibrahim  Abdel-Motaleb 

The  ultimate  goal  of  this  project  is  to  investigate  the  techniques  and  methodologies  by  which 
Engineering  students  can  learn  engineering  subjects  at  the  expert  level.  What  is  meant  by  “expert 
level”  is  that  the  students  learn  the  subject  scientific  bases,  applications,  and  limitations  that 
allow  them  to  reach  the  highest  level  of  Bloom’s  taxonomy:  synthesis  and  evaluation.  Reaching 
such  level  will  in  turn  allow  them  to  deal  with  any  problem  as  experts. 

To  achieve  the  above  goal,  several  learning  methodologies  are  investigated.  In  this  project,  the  impact  of 
cooperative  learning  on  engineering  students  is  investigated.  The  course  chosen  for  this  investigation  is 
the  three  credit  hours  course,  ELE  335:  Theory  of  semiconductors  devices  I.  The  course  covers  the 
properties  of  semiconductor  materials  and  the  theory  of  operation  of  semiconductor  devices,  including 
p-n  junctions,  bipolar  transistors,  and  field  effect  transistors.  To  conduct  the  investigation,  a custom 
model  is  developed  to  ensure  the  integrity  and  reliability  of  the  results. 


Custom  Model;  The  model  used  in  this  study  is  to  investigate  the  effectiveness  of  Individual 
Performance-Based  Eearning  vs.  Cooperative  Performance-Based  Eeaming.  The  basic  research 
question  in  this  model  is:  Does  individual  performance-based  learning  or  cooperative 
performance-based  learning  result  in  better  student  learning  as  reflected  in  an  end-of-unit  exam 
(either  midterm  or  final)?  The  plan  for  the  implementation  of  the  research  model  is  shown  in 
Table  I. 


Table  I.  Research  Model 


Individual  vs.  Cooperative  Performance-Based  Eearning 

Treatment 

Posttest 

Treatment 

Posttest 

Experimental 
Group  1 

Instruction 

Individual 
Performance-B  ased 

Midterm 

Instruction 

Cooperative 

Performance- 

Based 

Pinal 

Experimental 
Group  2 

Instruction 

Cooperative 
Performance-B  ased 

Midterm 

Instruction 

Individual 

Performance- 

Based 

Pinal 

Replication  1 

Replication  2 

Procedure: 

Students  will  be  randomly  assigned  to  the  two  experimental  groups  and  therefore  those  groups 
can  be  considered  equivalent.  The  experiment  will  actually  be  administered  twice  during  the 
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semester.  To  enhance  the  fairness  to  the  class  both  groups  will  be  exposed  to  both  treatment 
conditions  - alternating  across  the  two  replications.  The  final  is  not  a comprehensive  final  so 
there  should  be  no  carryover  from  the  midterm  to  the  final  that  might  contaminate  the 
interpretation  of  the  results. 

The  research  question  will  be  addressed  by  comparing  the  mean  test  scores  between  the  two 
groups  for  each  replication  of  the  experiment.  Two  performance  tasks  will  be  used.  Each 
performance  task  is  a design  project.  The  design  problems  are  related  to  the  subject  matter  of  the 
course,  but  are  not  directly  taught  or  discussed  during  the  lectures.  By  choosing  such  topics,  the 
project  topics  are  not  beyond  their  level,  but  at  the  same  time  they  require  more  research  to  be 
fully  comprehended  and  finished.  The  two  performance  tasks  are  assigned  as  follows: 

(A)  Pre-midterm  Performance  Task: 

Group- 1 : This  group  will  be  formed  in  subgroups  of  3-4  students,  and  they  will  be  required  to  present  a 
group  project. 

Group-2:  This  group  will  be  asked  not  to  collaborate  with  any  body  while  doing  their  project.  Each 
student  will  be  asked  to  submit  individual  project. 

(B)  Post-midterm  Performance  Task: 

Group- 1 : This  group  will  be  asked  not  to  collaborate  with  any  body  while  doing  their  project.  Each 
student  will  be  asked  to  submit  individual  project. 

Group-2:  This  group  will  be  formed  in  subgroups  of  3-4  students,  and  they  will  be  required  to  present  a 
group  project. 

In  this  study,  we  seek  to  answer  the  following  question: 

1-  Which  group  performed  better  in  the  midterm  and  the  final? 

2-  Can  the  improvement  of  performance  be  related  to  cooperative  learning? 

3-  Can  we  relate  the  students’  performance  to  the  timing  of  the  cooperative  learning?  Is  it  better  to 
have  co-operative  learning  at  the  beginning  or  at  the  end? 


(A)  Pre-midterm  Performance  Task  Project 

Students  are  required  to  design  a solid-state  device  that  is  not  covered  in  the  class,  but  is  based  on  the 
material  covered  in  the  class.  The  device  chosen  is  the  design  of  a laser  diode  that  can  emit  laser  at  600 
nm  wavelength.  In  this  course,  EEE  335,  Theory  of  Semiconductor  I,  lasers  are  not  covered,  but 
semiconductor  materials  and  p-n  junction  diodes  are  covered.  Since  laser  diodes  are  a form  of 
semiconductor  diodes,  the  topic  was  at  the  level  of  the  students,  but  students  needed  extra  research  work 
to  complete  the  design.  The  extra  work  is  expected  to  allow  them  to  reach  the  highest  level  of  Bloom’s 
taxonomy  con:  of  synthesis  and  evaluation. 

In  the  pre-midterm  project,  students  are  required  to  design  the  material  structure  for  the  laser  diode.  In 
this  case  they  have  to  choose  the  material  composition  with  an  energy  gap  that  produces  a wave  length 
of  600  nm.  The  students  realized  that  there  is  more  than  one  material  composition  that  can  produce  the 
required  wavelength.  The  students  realize  that  it  is  not  enough  to  have  a material  that  produces  the 
required  wavelength,  but  the  lasing  should  with  the  highest  efficiency.  If  the  material  structure  they  had 
chosen  did  not  result  in  high  efficiency,  they  should  figure  out  how  to  modify  the  structure  to  enhance 
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the  efficiency.  Because  these  issues  were  not  covered  in  the  course,  they  can  only  understand  them  from 
their  literatures’  search.  The  details  of  the  pre-midterm  performance  task  project  are  shown  in  Table  II 
below. 


Table  II:  Details  of  the  Pre-Midterm  Performance  Task  Project 

Pre-Midterm  project  topic:  Design  the  Material  structure  for  a 600  nm  Laser  Diode 
ABET  Criteria:  A,C,E,J,  and  H 
Student  learning  objectives 

1 . To  give  students  in  electrical  engineering  an  introduction  to  semiconductor  material  properties 

2.  To  learn  the  basic  theories  of  modem  electronic  devices 

3.  To  apply  semiconductor  theories  to  design  electronic  devices  and  investigate  their  performance 
Task  for  groups 

You  are  a member  of  a team  of  Engineers  who  are  charged  to  design  a laser  diode  to  operate  at  the  wave 
length  specified  above.  This  diode  will  be  manufactured  by  your  company.  In  this  project  you  need  to 
(a)  search  the  different  material  stmctures  of  laser  diodes  and  (b)  determine  the  best  material  structure 
and  composition  that  can  be  used  to  build  this  diode. 

You  are  required  to  do  the  following: 

(1)  search  the  literature  and  the  internet  to  learn  (a)  how  laser  diodes  are  built  and  (b)  what  the 
different  types  of  laser  diode  structures  are  available  in  the  literatures, 

(2)  select  the  material  stmcture  that  you  think  is  the  best  to  design  the  laser  diode  that  can  meet  the 
design  specifications  provided, 

(3)  adjust  the  material  composition  and  choose  the  substrate  type  to  ensure  that  (a)  the  different 
materials  are  lattice  matching  and  (b)  the  active  layer  will  have  the  required  band  gap  properties 
for  operation  at  the  above  wave  length, 

(4)  validate  your  choice  of  the  design  stmcture  analytically, 

(5)  compare  the  group  members  designs 

(6)  choose  a design  for  the  group  (you  may  modify  one  of  the  designs  or  adopt  a new  one), 

(7)  justify  the  choice  of  the  group  design,  and 

(8)  present  your  design,  analysis,  findings,  and  conclusion  as  report  by  the  deadline. 

As  a team:  (a)  Assign  specific  duties  to  each  member  of  your  research  team,  (b)  Keep  a log  of 
individuals  and  team  activities  and  progress  of  your  work,  (c)  As  you  work  together  you  will  find  you 
must  change  certain  things  to  make  your  team  work  effectively;  be  aware  of  those  behaviors  you  had  to 
change  and  what  you  did  to  change  them,  (d)  Document  the  changes  you  observed,  (e)  Show  how  you 
did  meet  the  challenges  resulting  from  these  changes,  (f)  Evaluate  the  performance  of  the  rest  of  the 
group  members. 

Task  for  Individuals 

You  are  charged  to  design  a laser  diode  to  operate  at  the  wave  length  specified  above.  This  diode  will  be 
manufactured  by  your  company.  In  this  project  you  need  to  (a)  search  the  different  material  structures  of 
laser  diodes  and  (b)  determine  the  best  material  structure  and  composition  that  can  be  used  to  build  this 


4 


diode/ 

Individually  you  are  required  to: 

(1)  search  the  literature  and  the  internet  to  learn  (a)  how  laser  diodes  are  built  and  (b)  what  the 
different  types  of  laser  diode  structures  are  available  in  the  literatures, 

(2)  select  the  material  structure  that  you  will  use  in  the  design  of  the  laser  diode  that  can  meet  the 
design  specifications  provided, 

(3)  adjust  the  material  composition  and  choose  the  substrate  type  to  ensure  that  (a)  the  different 
materials  are  lattice  matching  and  (b)  the  active  layer  will  have  the  required  band  gap  properties 
for  operation  at  the  above  wave  length,  and 

(4)  validate  your  choices  of  the  design  structure  analytically, 

(5)  justify  the  choice  of  the  group  design,  and 

(6)  present  your  design,  analysis,  findings,  and  conclusion  as  report  by  the  deadline: 


Rubric  used  in  the  performance  Task  evaluation: 

A rubric  was  created  to  evaluate  the  performance  of  the  students  in  the  pre-midterm  projects  is  shown  in 
Table  III.  For  individual  projects,  the  grading  points  were  divided  as  follows:  literature  review  20%,  the 
design  of  the  device  50%,  and  the  report  30%.  For  group  projects,  grading  points  were  divided  as 
follows:  literature  review  counted  20%,  the  design  of  the  device  40%  (10%  for  individual  work  within 
the  group  and  30%  for  the  work  of  the  group  as  a whole),  group  work  management  10%,  and  the  report 
30%. 


Table  III.  Rubric  for  performance  test  evaluation 


Criteria 

Level  of  Performance 

85-100% 

60-84% 

50-59% 

0% 

Literature 

review 

(20%  of  total) 

Major  references  are  refereed 
publications  and  books  and 
the  references  adequately 
cover  (1)  How  laser  diode  is 
built,  (2)  Types  of  laser  diode 
structures. 

References 
covers  the  two 
components 
before  but 
mainly  using 
non-refereed 
sources 

Insufficient 
coverage  of  one 
of  the 

components 

No  literature 
review  is  shown 

Individual 
design  Project 
(50%  of  total) 

(1)  The  right  material 
structure  is  chosen;  (2)  Right 
adjustments  to  the  material 
composition  are  done  (3) 
Right  substrate  is  chosen;  (4) 
Validation  for  the  adjustment 
is  presented. 

All  components 
are  shown,  but 
justification  is 
not  strong 

No  validation  is 
shown 

Unacceptable 

project 

Group  Design 
40% 

(Individual 
work  10%, 
group  30%) 

(1)  Individual  design  is 
evaluated.  (2)  Individual 
designs  are  compared.  (3) 
New  or  modified  design  is 
developed.  (4)  New  or 

All  components 
are  addressed, 
but  justification 
is  not  strong 

No  justification 
is  presented 

Unacceptable 
group  project 
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modified  design  is  justified. 

Group  work 
management 
(10%  of  total) 

(1)  Showing  detailed 
assignments  of  the  group 
members;  (2)  keeping  a log  of 
the  team  activities;  (3) 
behavior  changes  observed; 
(4)  documentation  of  the 
behavior  changes. 

One  component 
is  missing 

Two  component 
is  missing 

More  than  one 
component  is 
missing 

Report 
writing 
(30%  of  total) 

Report  is  well  written  and 
includes  (1)  individual 
designs,  (2)  group  design,  and 
(c)  other  components: 
abstract,  introduction,  design 
analysis,  results,  data  analysis, 
conclusions,  and  references. 

Individual 
design  and 
group  design  are 
well  written,  but 
there  some  of 
(c)  is  deficient. 

Poorly 

presented 

design 

component 

Unacceptable 
group  project 

Analysis 

The  average  score  on  the  Pre-midterm  performance  test  was  as  follows.  For  group- 1 (group  projects), 
the  average  score  of  the  project  is  91.33  %,  and  for  group-2  (individual  projects)  the  average  score  is 
only  71.7%.  This  means  the  average  score  of  the  students  with  group  projects  is  27.4%  higher  than  the 
average  score  of  the  students  with  individual  projects.  This  clearly  indicates  that  the  learning  outcome 
from  group  work,  or  cooperative  based  learning,  is  higher  than  the  individual  work,  or  the  individual 
based  learning.  The  question  that  needs  to  be  answered  is  whether  the  learning  outcome  achieved 
because  of  the  cooperative  learning  enhanced  the  overall  learning  outcome  of  the  course  materials,  or  it 
impacted  only  the  students’  knowledge  about  the  project. 

To  answer  this  question,  the  grades  of  the  midterm  are  analyzed.  The  average  midterm  score  for  students 
involved  in  group  projects  was  found  to  be  71.67%  compared  with  55.3%  for  students  involved  in 
individual  projects.  This  means  that  students  involved  in  group  projects  scored  on  average  29.57% 
higher  points  than  students  involved  in  individual  projects.  It  should  be  noticed  here  that  the  percentage 
increase  in  the  average  midterm  score  for  group  students  is  higher  by  almost  the  same  percentage  as  the 
average  score  of  the  pre-midterm  project  (29.5%  and  27.4%,  respectively).  This  indicates  that  the 
learning  level  enhancement  due  to  the  cooperative  learning  has  been  transferred  to  an  exact 
enhancement  of  the  learning  level  of  the  wider  course  subject  mater. 

(B)  Post-midterm  Performance  Task 

In  this  performance  task,  students  are  required  to  design  the  laser  diode  itself,  based  on  the  material 
structure  they  have  chosen  in  the  Pre-midterm  performance  task  project.  This  is  related  to  the  materials 
covered  in  the  second  half  of  the  course.  In  the  second  half,  solid  state  devices,  namely  diodes.  Bipolar 
Junction  Transistors  (BJTs),  and  Metal-Oxide-Field  Effect  Transistors  (MOSFETs),  are  covered. 
Therefore,  the  nature  of  the  Post-midterm  project  is  similar  to  the  nature  of  the  materials  covered  in  this 
period.  Since  laser  diodes  are  not  exactly  similar  to  regular  p-n  junction  diodes  covered  in  the  lectures,  it 
is  expected  that  the  Post-midterm  project  will  force  the  students  to  do  more  research  and  hence  allowing 
the  students  to  reach  the  highest  level  of  Bloom’s  taxonomy:  of  synthesis  and  evaluation.  The  details  of 
the  pre-midterm  performance  task  project  is  shown  in  Table  IV  below,  and  the  rubric  for  grading  is  the 
same  as  the  rubric  shown  in  Table  III. 
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Table  IV-  Post- Midterm  Performance  Task 

Post-Midterm  Project  Topic:  Design  the  a 600  nm  Laser  Diode 
ABET  Criteria:  A,C,E  J,  and  H 
Student  learning  objectives 

4.  To  give  students  in  electrical  engineering  an  introduction  to  semiconductor  material  properties 

5.  To  learn  the  basic  theories  of  modem  electronic  devices 

6.  To  apply  semiconductor  theories  to  design  electronic  devices  and  investigate  their  performance 

This  is  a continuation  of  the  Pre-Midterm  project  where  the  diode  will  be  designed  and  its  performance 
evaluated  at  the  above  wave  length.  The  material  structure  chosen  in  the  Pre-Midterm  project  must  be 
used  in  this  design. 

Individually,  you  are  required  to: 

(1)  search  the  literature  and  the  internet  to  (a)  learn  how  laser  diodes  are  built  and  (b)  understand 
how  the  device  performance  can  be  evaluated, 

(2)  use  the  laser  diode  material  structure  you  have  chosen  in  the  Pre-Midterm  project  to  design  your 
diode, 

(3)  simulate  the  diode  performance  and  fine  tune  the  material  structures,  composition,  and  doping  to 
ensure  the  diode  has  high  efficiency,  high  reliability  and  manufacturability,  and  low  cost,  and 

(4)  validate,  analytically  or  using  software,  that  your  design  will  best  meet  all  the  design 
requirements  mentioned  above. 

As  a team  you  are  required  to 

(1)  evaluate  and  compare  the  individual  structure  design, 

(2)  either  develop  a new  design  or  optimize  the  best  individual  design,  so  that  the  diode  will  best 
meet  the  design  specifications  mentioned  above,  structure  design  that  will,  most  likely,  result  in 
operating  at  the  specified  wave  length,  and 

(3)  justify  the  choice  of  the  group  design. 

As  a team:  (a)  Assign  specific  duties  to  each  member  of  your  research  team,  (b)  Keep  a log  of  team 
activities  and  progress  of  your  work,  (c)  As  you  work  together  you  will  find  you  must  change  certain 
things  to  make  your  team  work  effectively;  be  aware  of  those  behaviors  you  had  to  change  and  what  you 
did  to  change  them,  (d)  Document  the  changes  you  observed,  (e)  Show  how  you  met  the  challenges 
resulting  from  these  changes. 

Present  your  design,  analysis,  findings,  and  conclusion  as: 

■ The  report  from  this  activity  should  be  combined  with  the  Pre-midterm  report  to  form  one 
project. 

■ A group  report  should  contain  the  activities  done  by  each  student. 


Analysis 

The  grading  of  the  post-midterm  performance  task  project  shows  that  the  group- 1 (individual  projects) 
has  an  average  score  of  87.9%  while  grou-2  (group  projects)  has  a score  of  93.6%.  The  group  project 
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students  average  score  is  about  6%  higher  than  the  average  score  of  the  individual  project  students.  The 
average  score  for  both  groups  is  statistically  the  same.  The  invariance  between  the  scores  of  the  two 
groups  is  logical,  since  at  this  point  all  students  have  already  experienced  cooperative  learning.  Group- 1 
experienced  it  during  the  pre-midterm  project  and  group-2  during  the  post-midterm  project.  Therefore, 
their  learning  curves  of  all  students  should  be  the  same,  and  this  what  the  grades  of  the  post-midterm 
projects  show. 

The  question  that  needs  to  be  answered  is  whether  the  enhancement  of  the  learning  curve  shown  from 
the  scores  of  the  projects  has  been  transferred  to  an  enhancement  of  the  wider  knowledge  of  the  course 
materials  or  not.  The  final  examination  scores  can  give  a good  answer  to  this  question.  The  results  show 
that  group- 1 average  score  is  69.9  % and  group-2  an  average  score  is70.4%.  This  means  that  they  are 
statistically  the  same.  This  again  is  in  agreement  with  the  scores  of  the  projects  themselves.  This  means 
that  group- 1 learning  curve  has  been  enhanced  due  the  cooperative  learning  of  the  pre-midterm  project, 
while  group-2  learning  curve  has  been  enhanced  due  to  the  cooperative  learning  of  the  post-midterm 
project.  Since  every  group  has  experienced  group  learning,  they  should  have  the  same  level  of 
knowledge,  and  this  is  what  the  results  indicate. 

The  question  that  needs  an  answer  now  is  which  is  better:  to  do  the  cooperative  learning  project  at  the 
beginning  of  the  course  or  at  the  middle  of  the  course.  The  final  examination  appears  to  indicate  that  the 
timing  of  the  project  does  not  affect  the  learning  curve  of  the  students,  since  the  average  score  is  the 
same  for  both  groups.  Considering  the  results  from  both  the  midterm  and  the  final  exams,  it  appears  that 
group- 1 scores  are  higher  than  group-2  scores,  since  group- 1 scored  high  in  both  the  final  and  the 
midterm,  while  group-2  scored  higher  in  the  final  only.  This  also  means  that  the  average  score  for  all 
students  will  be  high,  as  long  as  they  start  their  cooperative  projects  at  the  same  time.  Starting  the  group 
projects  in  the  second  half  of  the  semester  may  be  attractive,  since  it  reduces  the  load  on  the  instructor, 
while  producing  the  same  results. 

From  the  above  discussion,  one  can  be  sure  that  the  learning  curve  of  group- 1 becomes  high  for  all  the 
course  materials  (pre-  and  post- midterms),  while  group-2  learning  curve  becomes  higher  only  in  the 
post-midterm  materials,  and  it  is  not  clear  what  is  their  level  in  the  pre-midterm  material.  To  investigate 
the  impact  of  the  timing  on  the  learning  level  for  all  course  material,  a take-home  examination  was  given 
to  all  students  at  the  end  of  the  course.  This  take  home  examination  contains  design  problems  that 
require  the  knowledge  of  all  the  course  materials.  The  results  show  that  the  average  score  of  the  take- 
home  examination  for  all  students  is  70.3%  which  is  exactly  the  same  as  the  average  score  for  the  final 
examination  and  the  post-midterm  project  for  both  groups.  These  results  may  suggest  that  the  timing  of 
the  project  is  not  critical.  However,  when  we  calculated  the  average  score  for  group- 1 separately  from 
group-2,  it  was  found  that  the  average  score  for  group- 1 is  80%  compared  with  52.2%  fro  group-2.  This 
shows  that  when  the  students  are  tested  for  all  course  materials,  the  group  that  did  the  project  at  the 
beginning  faired  much  better.  This  means,  cooperative  learning  affects  the  learning  level  during  and 
after  the  doing  the  group  project,  but  it  does  not  significantly  affect  the  learning  level  retroactively.  In 
other  words,  the  earlier  the  cooperative  learning  starts  the  higher  the  learning  curve  will  be. 

Conclusions 

From  this  study  one  can  conclude  the  following: 

1-  Cooperative  learning  results  in  significant  improvement  in  the  learning  performance  of  the 
students. 
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2-  Cooperative  learning  affects  the  learning  performance  during  or  after  the  group  work,  but  not 
retroactively. 

3-  To  ensure  the  best  results  for  all  the  students,  cooperative  learning  should  start  as  early  in  the 
semester  as  possible. 
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This  study  examines  the  effect  of  complex  performance  assessments  with  multiple  embedded  tasks  on  both  student 
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1.  Introduction 

There  have  been  many  excellent  publications  over  the  past  several  decades  discussing  how  to  improve  student 
learning.  Much  of  the  discussions  about  knowledge,  learning,  critical  thinking,  and  constructivism  directly  or 
indirectly  relate  to,  or  include,  active  learning  as  a requirement.  Bloom’s  Taxonomy  of  Educational  Objectives  [5]  is 
the  most  prominent  learning  taxonomy  in  use  in  education  today.  Deeper  knowledge  is  hopefully  achieved  as  one 
moves  to  the  higher  levels  (e.g.  synthesis  and  evaluation).  The  higher  levels  often  involve  active  learning 
participation.  Edgar  Dale’s  Cone  of  Learning  [9]  has  also  been  a standard  model  of  reference.  Active  learning 
occurs  when  students  talk,  listen,  read,  write,  and  reflect.  Problem  based  learning  (PBL)  and  training  in  problem 
solving  techniques  are  often  implemented  to  engage  the  students  in  active  learning.  Chickering  and  Gamson  wrote 
the  “Seven  Principles  for  Good  Practice  in  Undergraduate  Education.”  Principle  number  three  encourages  active 
learning,  and  states  that  “Learning  is  not  a spectator  sport...  Students  must  talk  about  what  they  are  learning,  write 
about  it,  relate  it  to  past  experiences,  and  apply  it  to  their  daily  lives.”  [8] 

The  emphasis  on  active  learning  has  occurred  due  to  the  fact  that  many  believe  it  fosters  better  retention,  as  well  as 
leads  students  to  expand  their  thinking  abilities.  [11]  Bransford’s  research  [6]  showed  that  students  were  more  likely 
to  retain  information  when  that  information  was  rehearsed  or  used  to  solve  problems.  Active  learning  implies  that 
the  student  is  actively  participating  in  the  learning  process.  If  this  is  done,  then  the  higher  levels  of  Bloom,  such  as 
synthesis  and  evaluation,  will  be  achieved.  Rebecca  Anderson  [1]  discusses  the  shift  from  traditional  assessment  to 
alternative  assessment.  Some  of  the  pertinent  beliefs  and  assumptions  of  traditional  assessment  are  that  it  treats 
learning  as  a passive  process  [13],  it  separates  process  from  product  [3],  and  assessment  is  objective  [2].  Alternative 
assessment  is  said  to  treat  learning  as  an  active  process  [17],  emphasize  process  and  product  [14],  and  view 
assessment  as  subjective  and  value-laden  [4].  Scarborough  [18]  suggested  a balanced  approach  to  assessment,  using 
a variety  of  tools  or  procedures.  This,  in  effect,  should  help  to  offset  limitations  of  one  type  of  assessment  against 
the  strengths  of  the  other. 

The  most  typical  form  of  assessment  during  much  of  the  past  century  in  the  United  States  has  been  the  use  of 
traditional  assessment,  and  more  specifically,  the  traditional  cognitive  test.  In  this  environment  the  learning  cycle 
most  likely  entails  the  student  attending  lectures  and  acquiring  information  from  the  “expert”,  as  well  as  from 
readings.  The  demonstration  of  knowledge,  or  assessment  technique,  would  then  be  in  the  form  of  a traditional 
cognitive  exam.  In  the  mid  1980’s  the  Western  world  started  to  consider  the  desire  to  foster  lifelong  learning,  and 
the  belief  that  fair  and  accurate  assessment  could  be  achieved,  through  “descriptive  review,”  for  tasks  demonstrating 
complex  products.  This  move  toward  performance  based  tasks  also  moves  the  student  towards  the  engagement  in 
active  learning. 


1 


Rahn  and  Moraga 


The  literature  is  rich  with  discussion  about  assessment  techniques  for  performance  based  tasks  and  the  need  for 
students  to  engage  in  active  learning.  Several  are  referenced  here.  [7,  10,  12,  15,  16,  19,  20,  21]  Even  though  many 
fine  methods  have  been  offered  for  assessing  performance  tasks,  many  in  the  fields  of  engineering  and  science  will 
no  doubt  argue  that  certain  key  fundamentals  must  be  mastered  and  tested  objectively,  with  assessments  such  as  the 
traditional  cognitive  test.  Unfortunately,  very  little  can  be  found  that  describes  or  discusses  the  effect,  if  any,  of 
students  participating  in  performance  based  tasks  and  the  ensuing  performance  on  traditional  cognitive  tests.  The 
work  presented  addresses  this  area  of  research.  This  paper  will  describe  a one  semester  study  of  a senior/graduate 
level  decision  theory  class  to  explore  if  participation  in  active  learning  helps  students  perform  better  on  traditional 
assessments,  and  if  retention  can  be  increased.  The  performance  tasks  the  students  worked  on  in  the  decision  theory 
class  used  for  this  study  achieved  high  levels  on  Bloom’s  taxonomy,  as  well  as  active  learning  according  to  Dale’s 
Cone.  They  were  required  to  apply,  synthesize,  and  evaluate  an  open-ended  system  of  their  choice  by  applying  the 
principles  of  decision  theory.  Oral  presentations  as  well  as  written  work  were  required  for  this  project. 

2.  Methodology 

This  study  examines  three  questions  with  regards  to  student  performance  on  traditional  tests  and  performance  tasks. 
The  underlying  procedure  for  all  questions  will  be  to  divide  the  students  into  two  randomly  chosen  groups.  One 
group  will  be  assigned  a performance  task  (Performance  Task  (PT)  #la)  in  which  decision  tree  methodology  is  to  be 
implemented  in  the  solution  to  the  task.  The  second  group  will  be  required  to  utilize  fault  tree  analysis  in  their 
solution  of  the  performance  task  (Performance  Task  #lb). 

The  three  questions  are  outlined  below.  The  corresponding  null  and  alternative  hypotheses  are  stated. 

1.  Does  a performance  based  task  covering  a subset  of  content  also  covered  in  a traditional  cognitive  test 
result  in  increased  learning  beyond  the  traditional  test  alone  as  indicated  by  the  traditional  cognitive  test? 

This  question  will  be  studied  by  using  the  following  procedure. 

• Each  student  was  assigned  randomly  to  one  of  two  groups.  One  of  these  will  be  termed  the 
experimental  group  and  the  other  the  control  group. 

• The  treatment  for  the  experimental  group  was  the  administration  of  the  traditional  test  (midterm 
exam)  along  with  a performance  task  (PT  #la),  which  included  activities  (decision  tree  analysis) 
covering  some  of  the  same  content  that  was  covered  in  the  traditional  test  (midterm  exam). 

• The  treatment  for  the  control  group  was  the  administration  of  the  traditional  test  (midterm  exam) 
and  a placebo  for  the  performance  task  (PT  #lb),  which  included  activities  still  vital  to  the  class 
(fault  tree  analysis)  but  not  covered  in  the  content  of  the  traditional  test  (midterm  exam). 

• After  the  administration  of  the  traditional  midterm  exam  (Posttest  1)  the  distinction  between  the 
groups  was  dissolved  with  instruction  and  assessment  activities  delivered  to  all  students  equally. 

Comparing  the  posttest  1 means  for  the  questions  regarding  decision  trees  across  both  the  experimental  and 
control  groups  will  address  this  question.  A t-test  for  significance  was  performed  and  the  significance  level  was 
determined.  A two-tailed  test  was  implemented. 

Null  Hypothesis  - Hq:  ^(experimental  group:  decision  tree)  = p (control  group) 

Alternative  Hypothesis  - Ha:  ^(experimental  group:  decision  tree)  ^ p (control  group) 

2.  Does  a performance  based  task  covering  a subset  of  content  also  covered  in  a traditional  cognitive  test 
result  in  increased  knowledge  retention  beyond  the  traditional  test  alone,  as  indicated  by  a final  exam? 

This  question  will  be  studied  by  using  the  following  procedure. 

• The  experimental  and  control  groups  remained  the  same  as  in  Question  1 . 

• The  treatment  for  the  experimental  group  was  the  administration  of  the  traditional  test  (final  exam) 
along  with  a performance  task  (PT  #la),  which  included  activities  (decision  tree  analysis)  covering 
some  of  the  same  content  that  was  covered  in  the  traditional  test  (final  exam). 

• The  treatment  for  the  control  group  was  the  administration  of  the  traditional  test  (final  exam)  and  a 
placebo  for  the  performance  task  (PT  #lb)  with  regards  to  decision  tree  content. 

After  the  administration  of  the  traditional  final  exam  (Posttest  2)  the  distinction  between  the  groups  was 
dissolved  with  instruction  and  assessment  activities  delivered  to  all  students  equally. 
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Comparing  the  posttest  2 means  for  the  questions  regarding  decision  trees  across  both  the  experimental  and  control 
groups  will  address  this  question.  A t-test  for  significance  was  performed  and  the  significance  level  was  determined. 
This  was  a two-tailed  test.  Figure  1 below  details  Questions  1 and  2. 

Null  Hypothesis  - Hq:  p(experimental  group:  decision  tree)  = p (control  group) 

Alternative  Hypothesis  - Ha:  ^(experimental  group:  decision  tree)  ^ p (control  group) 


Treatment 

Posttest  1 

Posttest  2 

Experimental 

Group 

Instruction 

Performance  Task  #la  - 
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A 
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Not  Related  to  Traditional 
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A 

Traditional  Test 
(Final  Exam) 

Figure  1:  Relationship  Between  Experimental  and  Control  Groups  for  Questions  1 and  2 

3.  Does  administration  of  a performance  based  task  affect  retention  on  a traditional  cognitive  test  (final  exam) 
compared  to  no  performance  assessment? 

This  question  will  be  studied  by  using  the  following  procedure. 

• Each  student  was  assigned  randomly  to  one  of  two  groups.  One  of  these  will  be  termed  the 
experimental  group  and  the  other  the  control  group.  The  students  will  remain  in  the  same  groups, 
the  only  difference  is  the  designation  of  the  experimental  and  control  groups. 

• The  treatment  for  the  experimental  group  was  the  administration  of  the  traditional  test  (final  exam) 
along  with  a performance  task  (PT  #lb),  which  included  activities  (fault  tree  analysis)  covering 
some  of  the  same  content  that  is  covered  in  the  traditional  test  (final  exam). 

• The  treatment  for  the  control  group  was  the  administration  of  the  traditional  test  (final  exam)  and  a 
placebo  for  the  performance  task  (PT  #la),  which  was  based  on  decision  tree  content.  This  group 
did  not  have  a performance  task  related  to  fault  tree  content. 

• After  the  administration  of  the  traditional  final  exam  (Posttest  2)  the  distinction  between  the 
groups  was  dissolved  with  instruction  and  assessment  activities  delivered  to  all  students  equally. 

Comparing  the  posttest  2 means  for  the  questions  regarding  fault  trees  across  both  the  experimental  and  control 
groups  will  address  this  question.  A t-test  for  significance  was  performed  and  the  significance  level  was  determined. 
This  was  a two-tailed  test.  Eigure  2 below  details  Question  3. 

Null  Hypothesis  - Hq:  ^(experimental  group:  fault  tree)  = p (control  group) 

Alternative  Hypothesis  - Ha:  ^(experimental  group:  fault  tree)  ^ p (control  group) 


Treatment 

Posttest 

1 

Posttest  2 

Experimental 

Group 

Instruction 

Performance  Task  #lb  - Group  having  a 
Performance  Task  with  fault  tree  content 

A 

Traditional  Test 
(Pinal  Exam) 

Control  Group 

Instruction 

Performance  Task  #la  - Group  not  having  a 
Performance  Task  with  fault  tree  content 

A 

Traditional  Test 
(Pinal  Exam) 

Figure  2:  Relationship  Between  Experimental  and  Control  Groups  for  Question  3 


3.  Analysis 

3.1  Data  for  Study  Question  1 

Does  a performance  based  task  covering  a subset  of  content  also  covered  in  a traditional  cognitive  test 
result  in  increased  learning  beyond  the  traditional  test  alone  as  indicated  by  the  traditional  cognitive  test 
(midterm  exam)? 

The  midterm  exam  contained  questions  covering  subject  matter  related  to  decision  trees,  and  not  to  fault  tree 
analysis.  The  methodology  for  the  calculation  of  the  significance  levels  is  critical.  Table  1 illustrates  the  data  for 
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the  midterm  exam.  The  experimental  group  participated  in  the  decision  tree  performance  task  (PT),  while  the 
control  group  participated  in  a fault  tree  PT.  The  total  points  possible  for  this  section  of  the  test  were  28  (with  100 
total  points  possible  for  the  entire  exam.) 


Table  1:  Scores  for  the  Traditional  Cognitive  Midterm  Exam 


Average  Score  on  Decision 
Tree  questions 

Variance  of  Score 
Decision  Tree  questions 

Experimental  Group 
(Decision  Tree  PT) 

18.28571 

1.16224 

Control  Group 
(Fault  Tree  PT) 

16.85714 

2.12244 

Note  that  the  scores  for  the  students  who  were  in  the  experimental  group,  those  who  participated  in  a performance 
task  covering  a subset  of  content  also  covered  in  the  traditional  cognitive  midterm  exam,  scored  higher  and  had  a 
lower  variance.  The  question  is  how  significant  is  this  difference?  The  calculation  of  the  variance  is  the  critical 
issue.  In  order  to  obtain  this  number  the  following  procedure  was  implemented: 

• A sample  variance  was  found  for  each  decision  tree  test  item  for  the  experimental  group. 

• Subsequently,  the  value  of  the  variance  of  the  sample  average  for  each  question  must  be  calculated.  To 
obtain  this  value  the  sample  variance  was  divided  by  the  number  of  students  in  the  group. 

• The  overall  variance  for  the  experimental  group  for  all  questions  related  to  decision  tree  subject  matter  can 
be  found  by  summing  all  of  the  variances  of  the  sample  averages  for  the  experimental  group. 

• This  was  then  repeated  for  the  control  group. 

The  significance  question  was  answered  by  performing  a t-test.  The  calculated  t-value  for  this  group  of  data  was 
0.739.  The  two-tail  area  for  this  t-value  came  out  to  0.462.  Consequently,  53.77  % of  the  time  one  would  expect 
the  scores  for  the  experimental  group  to  be  higher.  (Significance  = 53.77  %) 

3.2  Data  for  Study  Questions  2 and  3 

Does  a performance  based  task  covering  a subset  of  content  also  covered  in  a traditional  cognitive  test 
result  in  increased  knowledge  retention  beyond  the  traditional  test  alone,  as  indicated  by  a final  exam  (a 
traditional  test)?  For  this  question  the  experimental  group  is  the  same  as  in  section  3.1. 

Does  administration  of  a performance  based  task  affect  retention  on  a traditional  cognitive  test  (final  exam) 
compared  to  no  performance  assessment?  For  this  question  the  experimental  group  is  now  the  group  of 
students  who  participated  in  the  fault  tree  PT,  and  the  control  group  consists  of  those  participating  in  the 
decision  tree  PT. 

The  final  exam  contained  questions  covering  subject  matter  related  to  both  performance  tasks.  The  analysis  was 
performed  in  the  same  way  as  in  section  3.1.  Table  2 gives  the  data  for  the  final  exam  for  both  study  questions. 


Table  2:  Scores  for  the  Traditional  Cognitive  Final  Exam 


Average  Score 
on  Decision 
Tree  questions 

Variance  of  Score 
Decision  Tree 
questions 

Average  Score 
on  Fault  Tree 
questions 

Variance  of  Score 
Fault  Tree 
questions 

Experimental  Group 
Question  2 (Decision 
Tree  PT) 

25 

1.46938 

Control  Group 
Question  2 
(Fault  Tree  PT) 

22.71429 

2.79591 

Experimental  Group 
Question  3 (Fault  Tree 
PT) 

38.42857 

1.39463 

Control  Group 
Question  3 
(Decision  Tree  PT) 

35.71429 

1.82998 
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The  significance  level  information  for  both  study  questions  is  contained  in  Table  3. 


Table  3:  Data  Results  for  Research  Questions  2 and  3 


t-value 

Significance 

Level 

Study  Question  2 

1.107 

72.73% 

Study  Question  3 

1.512 

86.57% 

It  is  evident  from  Table  3 that  whichever  performance  task  the  student  groups  participated  in  corresponded  directly 
to  the  overall  performance  of  that  group  on  the  traditional  test.  The  performance  on  final  exam  test  items  related  to 
decision  tree  analysis  subject  matter  was  directly  correlated  to  the  performance  task  subject  matter.  This  also  held 
true  for  performance  on  final  exam  test  items  related  to  fault  tree  analysis  subject  matter. 

4.  Results  and  Conclusions 

This  paper  studied  (as  addressed  in  research  questions  1 and  3)  the  effect  of  complex  performance  assessments  with 
multiple  embedded  tasks  on  increased  learning  beyond  a traditional  test  alone  in  a senior/graduate  level  engineering 
decision  theory  class  as  indicated  by  traditional  cognitive  tests.  This  research  found  that  for  this  particular  class  a 
performance  assessment  task  covering  a subset  of  content  also  covered  in  a traditional  cognitive  test  resulted  in 
increased  learning.  This  was  evidenced  by  higher  scores  on  the  decision  tree  content  midterm  test  items  for  the 
experimental  group.  This  also  occurred  on  the  final  exam,  with  the  fault  tree  group  scoring  higher  on  the  fault  tree 
related  items  in  the  traditional  final  exam.  It  is  important  to  also  observe  that  the  variances  in  the  scores  were  lower 
for  the  experimental  groups,  meaning  that  the  scores  of  these  students  were  also  more  consistent  with  each  other. 

This  research  also  looked  at  the  knowledge  retention  beyond  the  traditional  test  alone,  as  indicated  by  traditional 
cognitive  test  (as  addressed  in  research  question  2).  This  was  done  by  tracking  the  student  scores  on  the  final  exam 
for  the  group  participating  in  the  decision  tree  PT.  The  data  also  points  to  an  increase  in  the  scores  for  these  students 
on  the  portion  of  the  final  that  dealt  with  decision  tree  subject  matter.  The  variance  for  these  students  was  also 
lower,  once  again  indicating  a more  consistent  performance.  Figures  3 and  4 illustrate  these  points  for  all  three 
research  questions. 

This  study  does  indicate  that  students  performing  active  learning,  via  a performance  task,  do  in  turn  translate  that 
into  better  performance  on  traditional  tests.  It  must  be  noted  that  additional  data  analysis  for  this  study  will  be 
performed  to  examine  the  effect  of  strength  of  student  bias  on  the  results.  In  other  words,  what  is  the  effect  of  an 
excellent  student  on  the  overall  results,  and  what  is  the  effect  of  a poor  student.  It  should  also  be  investigated  as  to 
what  level  of  additional  exposure  to  subject  matter  content  aids  in  increased  learning  and  retention.  Additional  work 
should  also  concentrate  on  looking  at  the  variance  component.  The  results  achieved  in  this  work  definitely  warrant 
further  research  into  this  area  of  student  learning. 
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Figure  3:  Scores  for  Experimental  Groups  Vs.  Control  Groups  for  All  Three  Questions 


5 


Rahn  and  Moraga 


Figure  4:  Variances  for  Experimental  Groups  Vs.  Control  Groups  for  All  Three  Questions 
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Initiative  Biographies 


The  group  of  professors  who  participated  in  The  Scholarship  of  Teaching  Initiative  were  a 
dedicated  faculty  group.  They  joined  the  Faculty  Learning  Community  with  open,  professional,  and 
enquiring  attitudes.  Their  humor  and  ability  to  critique  themselves  was  outstanding  throughout  the 
entire  process  of  critical  reflection  and  development.  Each  faculty  member  involved  in  this  pilot 
initiative  reflected  his  or  her  serious  interest,  dedication  to  teaching  and  learning,  professionalism, 
camaraderie,  and  leadership  without  question.  Very  important  to  the  program  leadership  is  that  they 
were  extremely  supportive  of  the  leaders.  As  one  of  those,  I must  say  that  I have  never  been  treated 
more  professionally  or  respectfully.  And  although  the  Initiative  activities  were  well  researched  and 
developed,  described,  discussed,  and  agreed  to  at  the  beginning  of  the  program,  it  was  impossible  to 
realize  at  that  time  what  the  full  extent  of  the  work  load  would  be  as  it  was  a pilot  program  and  our 
first  time  offering  the  program.  However,  each  and  every  faculty  participant  completed  everything 
asked  of  them;  they  were  tolerant  - allowing  aspects  of  the  program  to  develop  as  the  process  and 
content  evolved.  As  anyone  who  leads  professional  development  knows,  regardless  of  how  well  the 
content  is  planned,  even  when  based  upon  an  extensive  and  diversified  needs  assessment,  and  how 
well  all  the  materials  are  developed,  each  individual  and  group  collectively  is  different,  and  things 
become  known  that  were  not  beforehand,  or  adjustments  must  be  made  that  were  not  predictable,  or 
a program  component  is  needed  that  was  unexpected,  and  more.  The  most  significant  outcome  of 
the  experience  was  that  there  is  now  a group  of  eight  professors  who  would  like  to  continue  to  work 
together  on  teaching  and  learning.  Some  of  us  wondered  throughout  the  process  why  we  had  not 
worked  together  before.  That  outcome  is  what  we  hoped  would  occur. 

Promod  Vohra,  Ed.D.,  PE,  CSIT-  Electrical  Engineering.  Dean,  College  of  Engineering  and 
Engineering  Technology.  Well  published.  Outstanding  grant  awards  and  industrial  contracts. 

Regarding  this  Initiative,  the  most  important  points  to  make  about  the  Dean  of  the  College  are  about 
leadership.  He  has  a vision  that  fully  acknowledges  and  includes  The  Scholarship  of  Teaching  as 
Boyer  (1990)  intended  it  to  be,  integrated,  interactive,  interdisciplinary,  and  equal  to  all  other  types 
of  research:  discovery,  integration,  and  application.  Most  important  is  that  he  believes  in 
“Transforming  Eeadership”  as  defined  by  Burns  (1978)  and  “Superleadership”  as  defined  by  Manz 
and  Sims  (1989).  These  theories  are  all  about  transforming  others  into  leaders  or  acknowledging 
and  empowering  the  leadership  capabilities  in  others;  they  involve  the  development  of  leaders 
where  everyone  is  raised  to  higher  levels  intellectually,  morally,  and  ethically.  This  initiative  is  all 
about  leading  and  empowering  others;  that  is  the  heart  of  teaching  and  learning.  We  provide 
learning  experiences  where  our  students  achieve  their  highest  possible  potential  in  their  time  with  us 
and  go  on  to  achieve  more.  We  develop  their  intellectual  and  personal  capability  to  learn,  lead,  and 
ultimately  to  transform  their  world.  Dean  Vohra  has  elevated  “teaching”  to  its  rightful  status  among 
scholarly  activities  in  the  college.  He  has  empowered  the  program  leader,  acknowledged  her 
experience  and  capability  to  lead  a group  of  engineering  and  technology  faculty,  and  empowered 
the  faculty  members  to  formally  create  and  engage  in  a learning  community  that  will  sustain 
scholarship  on  teaching  for  the  future;  they  have  now  become  the  leaders  as  they  no  longer  need  the 
initial  leader.  Dean  Vohra  has  committed  to  sustaining  leadership  and  support  and  quietly 
empowered  his  faculty  in  the  true  sense  of  “empowerment”  where  one  gives  up  their  own  power  to 
empower  others  to  lead. 


Faculty  Participants 


Abul  K.M.  Azad,  Ph.D.  - Electrical  Engineering.  Associate  Professor,  Electrical  Engineering 
Technology.  Interests  are  logic  design,  microprocessing,  communication  systems  design,  digital 
communications.  Significant  Grants  awarded  by  the  National  Science  Eoundation,  the  U.S. 
Department  of  Education,  and  the  Engineering  and  Physical  Science  Research  Council,  UK.  Grant 
focii:  Robotics,  program  development,  and  analysis  of  railroad  barriers  for  transportation  planning. 
Well  funded  by  NIU  for  internal  grants:  web  based  laboratory  development,  design  and 
development  of  crowd  control  system  for  public  spaces,  internet  based  simulation  and  control  of 
environment;  intelligent  modeling;  artificial  intelligence  for  robot  manipulators;  collaborative 
interdisciplinary  simultaneous  engineering  project  development,  and  more.  Published  articles 
include  the  following  topics:  Internet-based  laboratory  experiments  in  engineering  technology; 
Design  and  development  of  a cost  effective  data  acquisition  system  using  PC’s  parallel  port; 
Internet-based  facility  for  physical  laboratory  experiments.  He  co-edited  Flexible  Robot 
Manipulators -Modeling,  Simulation  and  Control.  UK. 

Coller,  Brian,  Ph.D.  - Mechanical  Engineering.  Assistant  Professor,  Mechanical  Engineering. 
Industrial  experience  includes  Hughes  Aircraft  Company.  Interests  are  digital  control,  dynamic 
systems  and  control,  engineering  mathematics.  Grants  awarded  by  the  National  Science 
Eoundation.  Grant  focii  include  nonlinear  dynamics  of  triggering  controllers,  advanced  computing 
in  mechanical  engineering,  automotive  engineering,  ethanol  vehicles,  and  more.  Published  articles 
on  Intriguing  nonlinear  dynamics  of  a controller  with  a sluggish  actuator;  Surge/stall  interactions  in 
compressors;  Open  loop  control  of  planar  diffuser  flow;  A study  of  double  flutter;  Structural 
nonlinearities  and  the  nature  of  the  classic  flutter  instability;  Optimization  of  an  E85  powered 
Chevrolet  Silverado;  Vortex  model  for  control  of  diffuser  pressure  recovery;  Beneficial  actuator- 
induced  bifurcations  in  compressor  control;  Evolving  control  strategies  for  suppressing  heteroclinic 
bursting,  and  more. 

Abhijit  Gupta,  Ph.D.  - Mechanical  Engineering.  Associate  Professor  Mechanical  Engineering. 
Significant  Grants/industrial  contracts  awarded  on  vibration  analysis  of  exhaust  systems;  modal 
analysis  of  V229;  vibration  analysis  of  sensor  and  port;  analysis  and  design  of  pelican  guide  wheel 
suspension;  fatigue  behavior  of  foams.  Co-lead  on  acoustics  and  vibration  project.  Published 
articles  on:  finite  elements;  development  of  adaptive  algorithm  for  active  sound  quality  control; 
active  vibration  control  of  a structure  by  implementing  filtered-XEMS  algorithm;  a direct  method 
for  matrix  updating  with  incomplete  measured  data  and  without  spurious  modes;  effectiveness  of  a 
spray  able  damper  studied  using  multiple  test  methods;  electromagnetic  shock  absorbers;  a damage 
identification  method  using  vibration  modal  parameters  through  finite  element  discretization,  and 
more. 

Reinaldo  Moraga,  Ph.D.  - Industrial  and  Systems  Engineering.  Assistant  Professor  Industrial  and 
Systems  Engineering.  Interests  are  statistics  for  engineers,  operations  research,  industrial  control 
systems,  systems  simulation,  simulation  modeling  and  analysis,  discrete  systems  simulation, 
advanced  experimental  design  for  engineers.  Grants  awarded  on  application  of  operations  research 
and  management  science  models  in  disaster  operations  management;  modeling  and  simulation  of 
cost  considerations  throughout  design  processes;  design  of  heuristics  for  production  scheduling 
problems;  systems  models  and  process  optimization;  extensions  of  meta-RaPS  to  machine 


scheduling  problems;  effective  solution  approach  for  solving  the  0-1  multidimensional  knapsack 
problem;  simulation  modeling  and  analysis  of  space  shuttle  flight  hardware  ground  processing;  risk 
analysis  methodologies,  techniques,  and  tools  for  ground  operations.  New  professor;  grant 
proposals  in  process.  Patents:  Virtual  Reality  Interactive  Software  for  Teaching  in  Robotic  and 
Material  Handling  Systems;  Garden  Table.  Publications  on  using  system  dynamics,  neural  nets,  and 
Eigenvalues  to  analyze  supply  chain  behavior;  application  of  SCOR  to  E-Govemment;  detecting 
and  analyzing  patters  in  supply  chain  behavior;  disaster  and  prevention  management  for  the  space 
shuttle  during  lift-off;  heuristic  approaches  for  the  unrelated  parallel  machine  scheduling  problem 
with  machine-dependent  setup  times,  and  more.  Book  Chapters  include  Meta-heuristics:  A 
solution  Methodology  for  Optimization  Problems  in  Handbook  of  Industrial  and  Systems 
Engineering;  Technological  Proposal  for  Computer  Aided  Education:  Two  Cases  of  Virtual  Reality 
Applications  in  Gestion  de  la  Docencia  e Internacionalization  en  Universidades  Chilenas. 

Ibrahim  M.  Abdel-Motaleb,  Ph.D.,  PE  - Electrical  Engineering.  Professor  Electrical 
Engineering.  Interests  are  electrical  circuit  design  and  analysis;  semiconductor  devices  and 
fabrication;  integrated  circuits;  electrical  microsystems.  Grants  or  contracts  awarded  by  industry  on 
rapid  optimization  of  commercial  knowledge  for  U.S.  Army-Vehicles  Center  for  Bribology  and 
Coatings;  testing  and  evaluation  of  S&N  relay  for  use  in  earth  movement  vehicle;  nanofabrication 
education;  characterization  of  ZnSe/Ge  Hetrointerfacese  using  BioRads  SETS  system  and 
electrochemical  C-V  profiler;  investigation  of  nitrogen  doping  on  field  emission  device 
characteristics;  magnesium  diffusion  in  gallium  nitride;  characterization  of  magnesium  oxide  MOS 
capacitor;  zinc  eleenide  semiconductors;  integrated  microelectronic  systems;  also,  MEMS  sensors; 
nanotechnology;  material  growth  and  characterization;  device  fabrication  and  characterization; 
integrated  circuit  fabrication;  modeling  and  simulation.  Publications  on  modeling  and  simulation  of 
bipolar  junction  transistors  using  the  theories  of  thermodynamics;  characterization  of  ZnSe/Ge 
material  growth  using  the  Atomic  Eorce  Microscope;  non-quasi  static  modeling  of  HBT  junction 
capacitance;  and  more. 

Regina  DeMers  Rahn,  Ph.D.  - Nuclear  Engineering.  Assistant  Professor  in  Industrial  and  Systems 
Engineering.  Industrial  experience  at  GeneMetrix.  Interests  are  quality  control;  manufacturing 
systems;  six  sigma  performance  excellence;  modem  problem  solving;  productivity  and  modeling 
and  improvement;  process  capability;  process  modeling;  reliability;  engineering  statistics;  decision 
modeling.  Publications  on  potential  distribution  in  a remote  hollow  cathode  glow  discharge 
deposition  source;  technical  feasibility  studies,  tag  design  computer  code;  weighted  median  method 
for  centerline  estimation;  cost/throughput  improvement;  quality  improvement.  Director  of  Carter 
Program  for  Women  in  Engineering;  Research/development/industry  contracts  on  high  resolution 
software  for  performance  metrics;  new  methodology  for  job  shop  scheduling,  capacity  calculations, 
and  cost  estimations. 

Robert  Tatara,  Ph.D.  - Chemical  Engineering.  Assistant  Professor,  Plastics  Technology. 

Interests  are  chemical  qualities  of  plastics/composites;  manufacturing  processing  and  technology; 
heat  transfer;  fluid  mechanics;  engineering  thermodynamics;  experimental  methods  in  engineering; 
refrigeration  and  air  conditioning.  Research/industrial  contracts  on  manufacturing  potential  for 
ethanol  processing  residue  streams;  use  of  corn  processing  co-product  as  a biofiller  material  in  a 
plastic  resin;  thermoforming;  high  temperature  excursions  due  to  shear  heating,  during  injection 
molding  of  ABS  plastic  melts;  special  mode  inserts  to  lengthen  cooling  time  of  injection  molded 


plastic  parts  to  reduce  residual  stresses;  quantified  effeets  on  resin  melt  viscosities  of  filler  materials 
using  a capillary  rheometer;  plastic  processing  simulation  software  evaluation.  Co-Investigator 
“Experimental  Determination  of  the  effects  of  Oil  on  Heat  Transfer  in  Flooded  Evaporators.” 
Publications  on  analytical  and  experimental  studies  of  properties  of  ethanol  coproduct-filled 
plasties,  modeling  injeetion  molding  proeession  of  a polypropylene  closure  having  an  integral 
hinge;  measurement  of  forees  developed  by  a double-acting  mopneumatic  cylinder;  measurement  of 
spray  boiling  refrigerant  coefficients  in  an  integral-fin  tube  bundle  segment  simulating  a full  bundle; 
pool  boiling  of  pure  R134a  from  a single  Turbo-BII-HP  tube;  effects  of  oil  on  boiling  of 
replacement  refrigerants  flowing  normal  to  a tube  Bundle-(art  II.  R-I34a  and  123);  and  more. 

Jerry  Gilmer,  Ph.D.  - Educational  Psychology,  Measurement  and  Statistics.  Director  of  Testing 
Services,  NIU.  Program  Assoeiate.  Developed  and  taught  the  student  assessment  program 
eomponent  on  test  analysis  and  development.  Collaborated  with  Jule  Searborough  on  the  program 
and  classroom  research  design,  methodology,  and  statistical  procedures.  Developed  and  produced 
the  researeh  designs  and  statistical  reports  on  program.  Industrial  experience  as  measurement 
speeialist,  researeh  scientist  assistant,  and  direetor  of  Eaw  Programs  Test  Development. 

Gail  Jacky,  Director  of  NIU’s  Writing  Center.  Edited  the  Portfolio  with  great  patience  and  always  a 
smile. 

Jule  Dee  Scarborough,  Ph.D.  - Distinguished  Professor,  Emeritus.  Technology-Emeritus,  2007. 
Researched,  conceptualized,  developed,  directed,  and  taught  most  of  the  faculty  development 
program  and  research  semester  for  The  Scholarship  of  Teaching.  Produced  and  documented  all  of 
the  coneeptual  research;  designed  and  developed  all  the  program  process,  teaching  and  learning 
materials  (e.g.,  instruments,  feedback  and  evaluation  forms,  program  worksheets,  presentations, 
etc.).  Designed  program  and  elassroom  research,  methodology,  and  statistical  procedures  in 
eollaboration  with  Jerry  Gilmer.  Doeumented  and  produced  the  College  Portfolio.  Distinguished 
grants  and  teaching  professor.  W.K.  Kellogg  National  Eeadership  Fellow.  Studied  in  46  eountries 
for  extended  time.  Nationally  reeognized  and  awarded  for  research  , teaching,  and  professional 
activities.  Recognized  and  awarded  by  NIU.  Grant  awards  total  approximately  10  million  from 
National  Seienee  Foundation,  State  of  Illinois;  also,  additional  eontracts  with  business  and  industry. 
Aetive  with  business  and  industry  as  eonsultant,  grant  partner,  and  researeh  on  leadership  in 
manufacturing  sector.  Outstanding  Graduate  Professor.  Teaching  areas  include  Industrial  and 
Engineering  Management  and  Eeadership.  Program  Eeader  - Projeet  Management  Graduate 
Certificate  and  Master’s  Speeialization.  Twenty-five  years  teaehing  graduate  eourses  on  Total 
Quality  Management,  International  Business,  Industrial  Eeadership,  Advanced  Projeet 
Management,  High  Performance  Teaming,  and  undergraduate  senior  design  capstone;  also  16  years 
teaching  engineering/industrial  technology:  automated  manufacturing  (e.g.,  automated  systems, 
PECs,  automated  eontrollers,  senior  design  eapstone  eourse).  Twenty-three  years  working  with 
high  schools  to  improve  mathematics,  science,  and  technology  in  secondary  education.  Well 
published.  Retired  2006. 


PROFESSOR  KNOWLEDGE  ASSESSMENT 

(See  Research,  Portfolio  Section  A.3) 

Jule  Dee  Scarborough,  Ph.D.  and  Jerry  Gilmer,  Ph.D. 


Assessment  of  the  professor’s  knowledge  on  teaching  and  learning  served  as  one  of  the  research  and 
evaluation  variables  to  determine  the  success  of  the  CEET  faculty  development  program.  Therefore, 
a pretest  was  administered  before  each  program  component,  and  a post 
-test  was  administered  upon  the  completion  of  each  program  component.  With  some  program 
components,  however,  a third  assessment  was  administered,  as  we  were  interested  in  observing 
retention.  Also,  note  that  some  of  the  assessments  were  performance  based.  Eor  example,  once  the 
professors  were  tested  traditionally  on  test  analysis,  they  were  also  required  to  perform  two  test 
analyses  during  the  research  semester  on  the  new  midterm  and  final  exams.  All  seven  professors 
performed  the  midterm  analysis;  six  of  seven  performed  the  analysis  of  the  final  exam;  they  all 
prepared  a diagnostic  write  up.  The  performance  assessment  program  component  was  totally 
performance  based  with  the  baseline  being  that  none  of  the  professors  had  used  performance 
assessment  before.  See  each  individual  knowledge  assessment  analysis  below. 


Table  B.2.1;  Program  Knowledge  Content  Assessments 


Administration  1 

Administration  2 

Administration  3 

Administration  4 

See  pages  noted  for 
each  analysis  below  on 
following  pages. 

Statistically 
Significant  Gain 
(.05) 

Statistically 
Significant  Gain  (.05) 
or  performance 

Statistically  Significant  Gain 
(.05) 

Orientation  (p.2) 

Baseline 

Yes 

NA 

Yes 

Course  Analysis  (p.3) 

Baseline 

Yes 

NA 

NA 

Student  Learning 
Outcomes  (p.4) 

Baseline 

Yes 

Yes 

NA 

Test  Analysis  (p.5) 

Baseline 

Knowledge 
increased,  but  not 
statistically  sig.  -No 

Yes  and  Performance 

Yes  and 
Performance 

Performance 
Assessment  (p.6-13) 

There  was  no 
traditional  pretest, 
as  professors  had 
not  used 

performance  tasks 
or  rubrics- 
performance 
assessment; 
therefore,  they 
designed  and 
developed  3 
complex  tasks  and 
corresponding 
rubrics.  Baseline 
data  was  -0- 

Yes,  this  was 
performance  based 
assessment,  thus  no 
statistical  analysis.  A 
rubric  with  standards 
and  criteria  was  used. 
Professors  performed 
to  the  top  criteria  for 
each  standard  on  the 
rubric. 

None.  However,  the  scored 
rubrics  returned  to  each 
individual  student  in  each  of 
the  professors’  classes  were 
reviewed  to  determine  the 
quality  of  the  scored  rubrics 
and  feedback  to  students.  The 
rubrics  were  consistent  in 
scoring  and  appeared  to  be 
done  appropriately.  Professors 
were  so  impressed  with 
performance  assessment,  they 
all  reported  that  they  would 
continue  to  use  it  in  the 
experimental  course  and  their 
other  courses  as  well. 

NA 

Educational  Research 
(p.l4) 

Baseline 

Yes 

No 

NA 

Clearly,  the  ehart  above  reveals  that  the  CEET  Eaeulty  Development  on  Teaehing  and 
Eearning  resulted  in  knowledge  gain  by  the  professors  on  teaching  and  learning.  Eor 
individual  data  analyses,  see  each  description  below; 
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DISCUSSION  OF  PROFESSORS’  CONTENT  KNOWLEDGE  ON  PROGRAM 

ORIENTATION  CONCEPTS 

(see  Program  A. 5;  Instruments  in  C.l;  Worksheets  in  C.l  - Scarborough) 

Jerry  Gilmer,  Ph.D. 

An  assessment  of  the  professors’  general  knowledge  of  the  concepts  covered  during  the  program  orientation 
was  administered  prior  to  the  orientation  on  2/2/06  and  again  after  the  orientation  on  2/16/06.  On  Orientation 
Day,  the  Faculty  Development  Program  was  fully  explained  as  well  as  the  learning  process  to  be  used 
throughout  the  Program.  The  process  of  analysis,  active,  problem-centered,  and  performance  based  learning 
was  described.  Concepts,  primary  theories,  and  expected  products  were  explained  (e.g..  Teaching  Portfolios). 
Some  of  the  primary  theories  or  concepts  introduced  were  learning  styles,  teaching  models  and  styles, 
performance  assessment  and  rubrics,  test  analysis  and  development.  There  was  a strong  emphasis  on  the 
Scholarship  of  Teaching  and  the  National  Call  for  Action  by  the  Carnegie  Foundation  and  Boyer  (1990)  for 
research  on  teaching  and  for  educational  research  to  count  as  scholarship  alongside  the  other  three  types  of 
scholarship  - also  explained.  The  argument  about  what  professors  are  supposed  to  do,  teach  or  research  was 
mentioned.  In  addition,  the  concepts  of  Faculty  Learning  Communities  and  Circles  were  introduced  and 
explained.  The  assessment  was  intentionally  subjective  and  short  answer  so  professors  could  express  in  their 
own  words  what  they  absorbed.  The  maximum  number  of  points  from  the  assessment  is  23.  The  table  below 
contains  the  professors’  scores  for  both  administrations  of  the  assessment. 

Table  B.2.a.l 


Admin  1 
(2/2/06) 

Admin  2 
(2/16/06) 

Abdel-Motaleb 

0 

10 

Azad 

6 

12 

Coller 

9.5 

11 

Gupta 

1 

6 

Moraga 

2 

3.5 

Rahn 

5.5 

13.5 

Tatara 

2 

8 

Mean  = 

3.71 

9.14 

SD  = 

3.39 

3.52 

The  differences  between  the  means  across  all  seven  professors  from  the  different  administration  times  were 
analyzed  statistically  and  the  results  are  presented  in  the  table  below.  The  significance  levels  are  from  a paired 
samples  t-test  on  the  data  from  seven  professors  (df  = 6). 


Table  B.2.a.2 


Mean 

SD 

Difference  From 
Admin  1 

Sig.  Levei 

Admin  1 
(2/2/06) 

3.7 

3.4 

- 

- 

Admin  2 
(2/16/06) 

9.1 

3.5 

5.4 

.004 

The  data  in  the  table  indicate  that  the  increase  in  scores  between  the  two  administrations  was 
statistically  significant  at  the  .05  level  of  significance. 
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DISCUSSION  OF  PROFESSORS’  CONTENT  KNOWLEDGE  ON  COURSE 

ANALYSIS 

(see  Program  A. 5;  Instruments  in  C.2;  Worksheets  in  C.l  - Scarborough) 

Jerry  Gilmer,  Ph.D. 

An  assessment  of  the  professors’  general  knowledge  of  the  concepts  of  Course  Analysis  was  administered 
prior  to  the  presentation  of  the  concepts  on  2/2/06  and  again  after  the  presentation  of  the  concepts  on  2/16/06. 
The  Course  Analysis  process  involved  the  professors  in  analyzing  their  courses  and  instructional  practices  for 
a wide  variety  of  factors.  They  performed  analyses  to  determine  the  quality  of  their  course  content  (e.g., 
concepts,  theories,  information,  and  skills;  underlying  math  and  science  requirements  or  general  education 
needed  to  be  successful  in  their  course,  and  whether  the  content  was  factual,  conceptual,  procedural,  or  meta- 
cognitive).  This  aspect  of  the  analysis  also  involved  them  in  comparing  the  content  to  the  ABET/NAIT 
standards  and  examining  their  course  objectives  and  outcomes.  The  course  content  was  prioritized  into  three 
categories,  primary,  secondary,  and  important  to  mention;  they  reconsidered  or  added  a course  timeline  or 
schedule.  Furthermore,  the  professors  then  analyzed  their  courses  for  instructional  practices  or  teaching 
models  and  styles  and  student  learning  styles  to  determine  their  primary  practices,  determining  what  could  be 
changed.  They  also  analyzed  their  courses  using  Dale’s  Cone  of  Learning  and  Bloom’s  Taxonomy  of  Learning 
to  determine  the  level  of  engaged  or  active  learning  as  well  as  attempt  to  determine  levels  of  critical  thinking 
involved.  The  result  was  a GAPs  Analysis  Summary  showing  current  course  reality  and  identifying  a wide 
realm  of  changes  that  could  occur.  Finally,  another  aspect  of  the  analysis  process  was  to  examine  their  student 
learning  assessments,  identifying  strengths  and  weaknesses  through  test  analysis  and  then  making  the  decision 
to  add  performance  tasks  and  rubrics.  This  program  component  assessment  was  also  subjective  short  answer  to 
provide  opportunity  for  professors  to  express  what  they  absorbed  in  open-ended  expression.  The  maximum 
number  of  points  from  the  assessment  is  12.  The  table  below  contains  the  professors’  scores  for  both 
administrations  of  the  assessment. 

Table  B.2.b.l 


Admin  1 
(2/2/06) 

Admin  2 
(2/16/06) 

Abdel-Motaleb 

3 

3.5 

Azad 

2 

7 

Collar 

4.5 

6 

Gupta 

4 

4 

Moraga 

1 

3 

Rahn 

5.5 

9 

Tatara 

6 

8 

Mean  = 

3.71 

5.79 

SD=  1.82  2.34 

The  differences  between  the  means  across  all  seven  professors  from  the  different  administration  times  were 
analyzed  statistically  and  the  results  are  presented  in  the  table  below.  The  significance  levels  are  from  a paired 
samples  t-test  on  the  data  from  seven  professors  (df  = 6). 


Table  B.2.b.2 


Mean 

SD 

Difference  From 
Admin  1 

Sig.  Levei 

Admin  1 
(2/2/06) 

3.7 

1.8 

- 

- 

Admin  2 
(2/16/06) 

5.8 

2.3 

2.1 

.019 

The  data  in  the  table  indicate  that  the  increase  in  scores  between  the  two  administrations  was 
statistically  significant  at  the  .05  level  of  significance. 
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DISCUSSION  OF  PROFESSORS’  CONTENT  KNOWLEDGE  ON 
STUDENT  LEARNING  OUTCOMES 

(see  Program  A. 5;  Instruments  in  C.2;  Worksheets  in  C.l  - Scarborough) 

Jerry  Gilmer,  Ph.D. 

An  assessment  of  the  professors’  general  knowledge  of  the  concepts  in  Student  Learning  Outcomes 
was  administered  three  times:  prior  to  the  presentation  of  the  concepts  on  3/23/06,  after  the 
presentation  of  the  concepts  on  5/15/06,  and  again  at  the  conclusion  of  the  program  training  sessions 
on  5/24/06.  This  program  component  engaged  the  professors  in  further  analysis  of  their  current 
course  student  learning  objectives  or  outcomes.  This  involved  them  in  identifying  the  appropriate 
ABET/NAIT  standards  and  then  in  the  development  of  student  learning  outcomes.  As  part  of  the 
learning  process,  the  history  of  learning  objectives,  outcomes,  behavioral  objectives,  etc.,  and  the 
language  and  definitions  leading  up  to  the  use  of  learning  outcomes  were  presented  so  the 
engineering  and  technology  professors  could  grasp  the  meaning  and  controversy  more  deeply. 
Although  another  subjective  and  open-ended  short  answer  assessment,  the  assessment  did  require 
them  to  try  to  remember  some  definitions  and  meanings.  The  maximum  number  of  points  from  tbe 
assessment  is  13.5.  The  table  below  contains  the  professors’  scores  for  both  administrations  of  the 
assessment. 


Table  B.2.C.1 

Admin  1 

Admin  2 

Admin  3 

(3/23/06) 

(5/15/06) 

(5/24/06) 

Abdel-Motaleb 

2 

4.5 

7.5 

Azad 

1.5 

5 

4 

Collet 

2 

10 

9 

Gupta 

5.5 

6 

6.5 

Moraga 

1.5 

4 

4 

Rahn 

5.5 

9.5 

11.5 

Tatara 

4 

9.5 

10.5 

Mean  = 

3.14 

6.93 

7.57 

SD  = 

1.82 

2.64 

2.96 

The  differences  between  the  means  across  all  seven  professors  from  the  different  administration 
times  were  analyzed  statistically  and  the  results  are  presented  in  the  table  below.  The  significance 
levels  are  from  paired  samples  t-tests  on  the  data  from  seven  professors  (df  = 6). 


Table  B.2.C.2 


Mean 

SD 

Difference  From 
Admin  1 

Sig.  Level 

Admin  1 
(3/23/06) 

3.1 

1.8 

- 

- 

Admin  2 
(5/15/06) 

6.9 

2.6 

3.8 

.006 

Admin  3 
(5/24/06) 

7.6 

3.0 

4.4 

.003 

The  data  in  the  table  indicate  that  the  increase  in  scores  between  the  first  administration  and 
the  other  two  administrations  was  statistically  significant  at  the  .05  level  of  significance. 
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DISCUSSION  OF  PROFESSORS’  CONTENT  KNOWLEDGE  ON  TEST  ANALYSIS 

(see  Program  A. 5-  and  Instruments  in  C.2  - Gilmer) 

Jerry  Gilmer,  Ph.D. 

A knowledge  test  covering  the  test  analysis  content  was  administered  to  the  professors  four  times: 
immediately  prior  to  the  test  analysis  teaching  sessions  (2/16/06),  immediately  after  the  test  analysis 
sessions  (3/30/06),  immediately  after  the  conclusion  of  the  formal  teaching  sessions  (5/15/06),  and 
at  the  end  of  the  semester  during  which  the  professors  where  practicing  the  concepts  learned 
(12/15/06).  A short  review  of  the  test  analysis  content  occurred  just  prior  to  the  administration  of  the 
test  on  5/15/06.  The  maximum  number  of  points  from  the  test  is  10.  The  table  below  contains  the 
professors’  scores  for  each  administration  of  the  test. 


Table  B.2.d.l 

Admin  1 

Admin  2 

Admin  3 

Admin  4 

(2/16/06) 

(3/30/06) 

(5/15/06) 

(12/15/06) 

Abdel-Motaleb 

3 

4 

7 

4 

Azad 

3 

7 

4 

6 

Coller 

2 

10 

10 

8 

Gupta 

4 

4 

7 

7.5 

Moraga 

6 

3 

7 

7 

Rahn 

2 

10 

8 

8 

Tatara 

5 

6 

5 

8 

Mean  = 

3.6 

6.3 

6.9 

6.9 

SD  = 

1.5 

2.9 

1.9 

1.5 

The  differences  between  the  means  across  all  seven  professors  from  the  different  administration 
times  were  analyzed  statistically  and  the  results  are  presented  in  the  table  below.  The  significance 
levels  are  from  paired  samples  t-tests  on  the  data  from  seven  professors  (df  = 6). 


Table  B.2.d.2 


Mean 

SD 

Difference  From 
Admin  1 

Sig.  Level 

Admin  1 
(2/16/06) 

3.6 

1.5 

- 

- 

Admin  2 
(3/30/06) 

6.3 

2.9 

2.7 

.134 

Admin  3 
(5/15/06) 

6.9 

1.9 

3.3 

.025 

Admin  4 
(12/15/06) 

6.9 

1.5 

3.4 

.005 

The  data  in  the  table  indicate  that,  although  the  professors’  knowledge  in  test  analysis 
concepts  increased  from  the  first  administration  to  the  second,  the  increase  was  not  statistically 
significant  at  the  .05  level  of  significance.  However,  the  professors’  knowledge  gains  after  the 
5/15/06  review  and  after  they  had  an  opportunity  to  apply  the  knowledge  in  their  own 
classrooms  were  statistically  significant  compared  to  the  first,  pre-teaching,  administration. 
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DISCUSSION  OF  PROFESSORS’  CONTENT  KNOWLEDGE  ON 
PERFORMANCE  ASSESSMENT 
PERFORMANCE  TASK  AND  RUBRIC  DEVELOPMENT 


(see  Program  A. 5 and  Instruments  in  C.2  - Scarborough) 

(pages  6-13;  Educational  Research  follows  on  page  14) 

Jule  Dee  Scarborough,  Ph.D. 

The  learning  and  professional  growth  on  the  Performance  Assessment  program  or 
knowledge  component  was  measured  by  the  professors’  performance  on  the  task  of 
designing  and  developing  three  complex  performance  tasks  and  three  corresponding  rubrics 
for  scoring  task  achievement.  Using  the  Rubrics  below  as  guiding  criteria,  they  each 
designed  three  complex  performance  tasks  and  corresponding  rubrics.  These  assessments 
were  added  to  their  course  as  new  assessment  strategies  and  assessment  procedures. 

It  is  important  to  note  that  one  performance  task/rubric  was  designed  to  correspond  with  the 
midterm  and  another  to  correspond  with  the  final  exam  using  the  logic  that  objective  tests 
usually  reflect  what  students  know  or  know  about  rather  than  what  they  can  do.  Therefore, 
we  used  an  unusual  scenario  where  the  professors  “linked”  the  objective  midterm  exam  to  a 
midterm  performance  task/rubric  and  an  objective  final  exam  to  a final  performance 
task/rubric.  They  also  developed  a third  performance  task/rubric  and  choose  how  and  when 
to  use  it.  They  were  asked  to  “match”  where  they  thought  the  test  items  and  performances 
“overlapped”  and  measured  the  same  or  similar  content.  It  was  assumed  from  studying  the 
literature  that  performance  assessment  measures  different  aspects  of  learning,  sometimes 
deeper  levels  of  learning  through  use  of  knowledge  in  more  active  or  engaging  ways, 
problems,  projects,  etc.  But  performance  assessment  can  also  measure  some  of  the  same 
aspects  of  learning  as  objective  tests.  Also  some  of  the  professors  designed  their  tests  to 
incorporate  some  level  of  performance  in  subjective  or  problem-based  items.  In  examining 
the  tests  and  analyzing  them,  the  objective  items  were  separated  from  the  more 
performance-based  items.  Professors  were  provided  a presentation  about  Performance 
Assessment.  Performance  Tasks  and  Rubrics  were  discussed,  and  they  received  a portfolio 
of  sample  tasks  and  rubrics.  They  were  given  books  on  the  topic  as  part  of  their  new  library 
on  teaching  and  learning. 

Their  performance  tasks  and  rubrics  reflect  the  ABET  or  NAIT  standards  with 
corresponding  rubrics.  Perhaps  one  professor  had  used  simple  and  less  formalized  rubrics 
before,  but  none  of  the  professors  had  developed  or  used  formal,  written,  scenario-based 
performance  tasks  with  corresponding  rubrics  before  this  initiative.  Thus,  there  were  no 
previous  instruments  to  view  from  the  baseline  semester.  Fall  2005,  and  compare  to  these. 
Therefore,  we  judged  them  based  upon  the  Rubrics  below. 

Performance  Task:  Design  and  develop  three  complex  performance  tasks  with 
corresponding  rubrics.  The  tasks  must  be  based  upon  the  ABET  outcomes  or  NAIT 
standards  and  corresponding  student  learning  outcomes  for  the  course;  they  must  also 
reflect  real  world,  authentic  performances  or  tasks  in  the  appropriate  community  of  practice, 
e.g.  industry.  The  performance  tasks  and  rubrics  must  be  used  to  measure  student  learning 
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in  the  experimental  research  course,  Fall  2006.  See  the  Rubrics  below  for  the  achievement 
criteria  to  use  in  accomplishing  the  task. 

Professors’  Performance;  The  professors  accomplished  the  performance  task  well.  The 
process  involved  drafting  initial  and  authentic  real  world  scenarios  with  embedded  task 
clusters  and  a corresponding  rubric  instrument  for  each  task  (3).  The  program  leader 
provided  feedback  one-on-one  as  the  performance  tasks  were  developed.  The  professors 
shared  their  drafts  with  each  other  and  benefited  from  the  group  critique  process.  The  group 
process  worked  especially  well.  The  tasks  and  rubrics  were  finalized;  the  program  leader 
approved  them;  and  then,  each  professor  used  the  tasks  and  corresponding  rubrics 
successfully  with  students  during  the  2006  experimental  research  semester.  After  the 
semester  was  completed,  the  professors  copied  all  rubrics  returned  to  each  student  in  their 
classes  for  all  three  performance  tasks.  The  program  leader  reviewed  the  scored/with 
comments  rubrics  that  each  student  received  back  from  the  professors.  Thus,  the  use  of  the 
rubrics  was  also  reviewed.  Finally,  the  professors  completed  a feedback/evaluation  form 
about  the  use  of  performance  assessment  for  the  first  time.  As  with  test  analysis  and 
development,  the  feedback  from  the  professors  on  the  value  of  learning  to  design,  develop, 
and  use  performance  tasks/rubrics  was  extremely  positive. 

The  following  rubrics  were  used  to  guide  the  professors  in  the  development  of  the  three 
performance  tasks  and  corresponding  rubrics  for  each  task. 

Also,  the  feedback  and  evaluation  questionnaire  and  professor  responses  are  provided 
below,  following  the  rubrics.  The  faculty  members  truly  felt  that  expanding  their 
assessments  to  include  performance  tasks  with  rubrics  was  extremely  positive.  They  all 
indicated  that  they  will  continue  to  use  performance  assessment,  tasks  and  rubrics,  and  also 
expand  the  use  of  performance  assessment  to  other  courses. 
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Rubric  for  Assessing  the  Quality  of  a Performance  Task 

Key  Components  - Properly  Designed  Performance  Tasks  must 

I.  Be  based  on  content  standards  established  by  ABET  or  NAIT 

II.  Describe  a “real-life”  scenario;  are  real  world,  authentic  tasks;  require  active  performances 

III.  Involve  students  in  complex  reasoning  - critical  thinking  at  upper  levels  of  Bloom’s  Cognitive  Dimension 

IV.  Require  students  to  collect  and  process  information,  using  it  for  an  authentic  purpose 

V.  Incorporate  “habits  of  mind” 

VI.  Require  student  collaboration  and  cooperation;  incorporates  “individual”  and  “group”  learning  and  performance 
accountability 

VII.  Result  in  a tangible  product  and/or  communication  activity 

For  each  component,  there  are  descriptors  reflecting  levels  of  achievement  possible: 

I.  The  Performance  Task  is  based  on  the  ABET  or  NAIT  standards 

a.  The  Performance  Task  is  directly  related  to  the  ABET  or  NAIT  standards. 

b.  Learning  standards  are  apparent,  but  the  relation  to  the  task  and/or  national  standards  is  sketchy  or  not  apparent. 

c.  The  Performance  Task  does  not  appear  to  be  based  on  the  standards/outcomes,  course  or  national. 

II.  The  Performance  Task  describes  a “real-life”  scenario  that  is  authentic  and  requires  active  performance. 

a.  The  scenario  described  in  the  task  accurately  mirrors  an  activity  in  the  community  of  practice  outside  the  classroom. 

b.  The  scenario  described  in  the  task  simulates  an  activity  in  the  community  of  practice  outside  the  classroom. 

c.  The  scenario  described  in  the  task  contains  some  aspects  of  activity  outside  the  classroom  but  is  largely  contrived. 

d.  The  scenario  described  in  the  task  is  an  academic  exercise  that  usually  takes  place  only  in  the  context  of  an  academic 
setting. 

III.  The  Performance  Task  involves  students  in  complex  reasoning-critical  thinking  processes  at  upper  levels  of  Bloom’s 
Cognitive  Dimension. 

a.  The  task  requires  students  to  utilize  complex  reasoning  - critical  thinking  skills,  such  as  induction/deduction,  diagnosis, 
abstracting,  experimental  inquiry,  problem  solving;  evaluation,  creation,  synthesis,  etc. 

b.  The  task  requires  students  to  utilize  complex  reasoning  components,  such  as  comparing,  classifying,  decision  making, 
or  investigation. 

c.  The  task  requires  students  only  to  recall  facts. 

IV.  The  Performance  Task  requires  students  to  collect  and  process  information,  using  it  for  an  authentic  purpose. 

a.  The  task  incorporates  a variety  of  information  gathering  techniques  and  information  resources.  Students  are  required  to 
interpret  and  synthesize  information  and  accurately  assess  the  value  of  information  gathered.  They  are  required  to 
collect  the  right  information  for  an  authentic  purpose,  e.g.  solve  a problem,  apply  or  use  in  a complex  project,  etc. 

b.  The  task  requires  students  to  gather  and  synthesize  information,  but  the  value  of  the  information  gathered  is  not 
assessed.  Information  may  not  be  used  for  a purpose. 

c.  The  task  requires  the  students  to  gather  information,  but  not  to  interpret  it. 

d.  The  task  requires  no  gathering  or  processing  of  information. 

V.  The  Performance  Task  incorporates  “Habits  of  Mind.” 

a.  The  task  requires  students  to  make  effective  plans,  use  necessary  resources,  evaluate  effectiveness  of  their  own  actions, 
seek  accuracy,  and  engage  in  activities  when  answers  or  solutions  are  not  immediately  apparent. 

b.  The  task  only  requires  students  to  effectively  plan  or  use  resources. 

c.  The  task  does  not  require  students  to  engage  in  self-regulation,  critical,  or  creative  thinking. 

VI.  The  Performance  Task  requires  student  collaboration  and  cooperation;  incorporates  “individual”  and  “group” 
learning  and  performance  accountability. 

a.  The  task  requires  students  to  use  interpersonal  skills,  work  toward  the  achievement  of  team  goals,  and  perform  a variety 
of  roles  within  the  team.  There  is  a formal  team  structure  and  process. 

b.  The  task  requires  students  to  work  together  in  teams  but  there  are  no  measures  described  that  ensure  collaboration  or 
cooperation  among  team  members. 

c.  The  task  is  completed  largely  by  students  on  an  individual  basis  rather  than  in  student  teams. 

VII.  The  Performance  Task  results  in  a tangible  product  and/or  communication  activity. 

a.  The  task  result  is  a tangible  product  or  communication  activity  comparable  to  that  commonly  produced  in  business  or 
industry  community  of  practice. 

b.  The  task  results  in  a product  that  is  similar  to  those  completed  in  business  or  industry  community  of  practice,  but  lacks 
several  components  that  make  the  product  realistic. 

c.  The  task  does  not  result  in  a product  or  communication  activity  relevant  to  a business  or  industry  community  of  practice. 
(Scarborough,  2006  [Based  upon  White  & Scarborough,  20041) 
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Rubric  for  Assessing  the  Quality  of  a Rubric 


Properly  Designed  Rubrics  Must 

I.  Contain  a set  of  key  components/standards  to  be  assessed  that  reflect  the  student  learning  outcomes 
for  the  course,  which  are  directly  linked  to  the  national  outcomes. 

II.  Include  descriptors  for  each  component/standard  that  are  measurable. 

III.  Have  descriptors-criteria  that  are  indicative  of  observable  student  performances  or  behaviors. 

IV.  Incorporate  a clear  and  well-defined  scoring  system 

V.  (Optional)  Include  appropriate  weights  for  each  component  and  descriptor 

For  each  component,  there  are  descriptors  reflecting  levels  of  achievement  possible: 

I.  The  rubric  contains  a set  of  key  components  (standards)  to  be  assessed. 

a.  A complete  list  of  key  components-standards  is  provided  for  the  performance  task,  including  the  embedded 
subtasks,  if  a cluster.  The  task(s)  are  directly  connected  to  student  learning  outcomes  for  course  and  the  national 
outcomes. 

b.  Key  components/standards  listed  are  not  exhaustive  for  the  performance  task  and/or  subtasks  embedded  are  not 
clear  enough  for  student  response  or  action;  components  or  standards  are  not  clearly  connected  to  student  learning 
outcomes  for  course. 

c.  Not  all  key  components/standards  describe  student  outcomes;  some  are  not  directly  linked  to  national 

outcomes. 

d.  No  key  components  are  listed. 

II.  The  rubric  includes  a set  of  descriptors-criteria  for  each  key  component  or  standard. 

a.  Descriptors-criteria  for  each  component  or  standard  are  arranged  in  a clear  hierarchy  from  non-achievement  to  full- 
achievement. 

b.  Descriptors-criteria  are  present  for  each  component/standard,  but  obvious  levels  in  some  are  missing. 

c.  Each  component  does  not  have  an  associated  set  of  descriptors-criteria. 

III.  The  rubric  descriptors/criteria  are  clear  and  contain  observable  or  measurable  student 
performances  or  behaviors. 

a.  All  descriptors-criteria  clearly  delineate  levels  of  observable  student  performances  or  behaviors. 

b.  Most  descriptors-criteria  clearly  delineate  levels  of  observable  student  performances  or  behaviors. 

c.  Only  a few  descriptors-criteria  clearly  define  levels  of  observable  student  performances  or  behaviors. 

d.  Descriptors-criteria  do  not  describe  observable  student  performances  or  behaviors. 

IV.  Incorporate  a clear  and  well-defined  scoring  system 

a.  There  is  a well  defined  and  clear  system  for  scoring  each  component-standard  and  its  descriptors- 

criteria.  Points  or  percentages  are  assigned  appropriate  to  instructional  and  performance  values. 

b.  The  scoring  system  lacks  definition,  clarity,  and  although  there  is  a scoring  system,  some  aspects  are 
ambiguous,  subjective  or  unclear. 

c.  There  is  no  scoring  system. 

V.  Optional:  Appropriate  weights  are  assigned  to  components  and  descriptors. 

a.  Component-standards  and  descriptors-criteria  are  each  properly  weighted  according  to 

instructional  emphasis  and  performance  values. 

b.  Weights  are  assigned,  but  point  values  do  not  reflect  proper  instructional  emphasis  or  performance  values  in  all 
cases. 

c.  Weights  are  assigned  to  some  performance  standards  and  descriptors,  but  not  others. 


(Scarborough,  2006  [Based  upon  White  & Scarborough,  2004]) 
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CEET  Initiative  on  Teaching  and  Learning  - Performance  Assessment  Feedback,  Jan, 

2007 

(7/7  respondents) 

1.  Was  the  time  spent  developing  performance  tasks  worth  your  time  - worthwhile? 

(7)  Yes  Not  really 

Why? 

• “A  lot  of  work,  but  they  really  engage  students” 

• “Allowed  me  to  think  about  what  students  should  be  able  to  perform  after  completing 
the  course.  ” 

• “Invaluable.  ” “Although  I always  give  ‘projects  I was  naive  to  many  of  the  aspects  of 
a true  performance  task.  ” 

• “Performance  tasks  made  students  (1)  solve  open  ended  problems;  (2)  work  in  groups; 
(3)  identify  problems  and  try  to  have  multiple  solutions  and  then  justify  the  solution.  ” 

2.  Was  the  time  spent  developing  rubrics  for  scoring  the  performance  tasks  worth  your  time  - 
worthwhile? 

(7)  Yes  Not  really 

Why? 

• “Students  knew  what  was  expected  of  them;  grading  was  a bit  easier;  fewer  students 
challenged  their  grades.  ” 

• “Allowed  me  to  set  expectations  from  the  PAs.  ” 

• “The  students  really  responded  well  to  them!  They  liked  knowing  the  expectations  for 
performance  tasks.  ” 

• “1  didn  ’t  have  this  experience  before.  ” 

• “Rubrics  helped  students  understand  what  is  expected  of  them  and  how  they  will  be 
graded.  ” 

3.  Would  you  recommend  the  performance  task  program  content  for  other  faculty  members? 

(7)  Yes  Not  really 

Why? 

• “Valuable” 

“It  will  be  a good  exercise  for  others.  ” 

• “I  think  any  new  faculty  should  be  exposed  to  this  experience.  ” 

• “1  feel  students  learned  a lot  because  of  the  performance  tasks.  ” 
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4.  Would  you  recommend  the  rubric  program  content  for  other  faculty  members? 


(7)  Yes  Not  really 

Why? 

• “It  will  be  a good  exercise  for  others.  ” 

• “This,  I believe,  is  a necessity.  ” 

5.  Were  the  performance  tasks  a beneficial  addition  to  the  student  assessment  plan  for  your 
course? 

(7)  Yes  Not  really 


Why? 

• “It  was  another  dimension  of  assessment.  Some  students  who  did  not  do  well  on 
tests. . . really  shined  in  performance.  ” 

“Provides  additional  form  of  assessment  method.  ” 

• “It  added  a new  dimension  of  student  assessment;  also  these  performances  tasks 
involved  various  learning  styles.  ” 

6.  Were  the  rubrics  beneficial  for  scoring  the  performance  tasks? 


(7)  Yes 


Not  really 


Why? 

• “Students  knew  what  was  expected  of  them;  grading  was  a bit  easier;  fewer  students 
challenged  their  grades.  ” 

• “Makes  the  scoring  process  easier.  ” 

• “It  made  it  easier  for  me  — and  the  students  also  responded  well.  IT  is  necessary  to  have 
a procedure  mapped  out  for  them  to  understand  the  expectations  and  levels.  ” 

• “They  make  grading  progress  easier.  ” 

• “Otherwise,  it  would  be  very  subjective  or  arbitrary.  ” 


1.  Were  the  performance  tasks  an  effective  tool  for  enhancing  student  learning? 


(7)  Yes 


Not  really 


Why? 

• “Bigger,  more  authentic  tasks.  ” 

• “Students  can  demonstrate  what  they  can  perform  after  completing  the  course.  ” 

• “(l)It  allowed  for  many  more  teaching  styles  to  be  incorporated  in  the  course;  (2)  more 
learning  styles  were  also  included;  (3)  a good  tool  for  group  work  as  well.  ” 

• “They  really  understand  expectations.  ” 

• “Students’  learning  involves  various  learning  and  teaching  style,  and  models  and 
performance  tasks  provided  these  opportunities.  ” 
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8.  Were  the  performance  tasks  an  effective  tool  for  measuring  student  learning? 

(7)  Yes  Not  really 

Why? 

• “It  was  another  dimension  of  assessment.  Some  students  who  did  not  do  well  on 
tests. . . really  shined  in  performance.  ” 

• “Students  can  demonstrate  what  they  can  perform  after  completing  the  course.  ” 

• “(1)  It  demonstrated  their  abilities  to  communicate  effectively;  (2)  It  demonstrated  their 
abilities  to  synthesize,  apply,  and  evaluate  subject  matter  content.  ” 

• “Students’  learning  may  not  be  completely  assessed  by  only  exams  and  homework.  ” 

9.  Were  the  rubrics  an  effective  tool  for  scoring  the  outcomes  of  student  performances  on  the 
tasks? 

(7)  Yes  Not  really 

Why? 

• “It  was  another  dimension  of  assessment.  Some  students  who  did  not  do  well  on 
tests. . . really  shined  in  performance.  ” 

• “See  previous  comments.  ” 

• “Rubrics  provide  the  details  of  expected  outcomes.  ” 

10.  Were  the  rubrics  effective  in  helping  students  to  understand  more  about  what  you 
expected  them  to  do  by  revealing  the  standards  and  scoring  mechanism  with  them  up  front? 

(7)  Yes  Not  really 

Why? 

• “It  was  another  dimension  of  assessment.  Some  students  who  did  not  do  well  on 
tests. . . really  shined  in  performance.  ” 

• “Students  know  the  expectations.  ” 

• “See  previous  comments.  ” 

• “They  know  what  is  expected  of  them.  ” 

11.  Do  you  feel  that  more  formalized  performance  tasks  and  rubrics  improve  the  opportunity 
for  students  to  provide  evidence  of  learning? 

(6)  Yes  (1)  Not  really 

Why? 

• “It  was  another  dimension  of  assessment.  Some  students  who  did  not  do  well  on 
tests. . . really  shined  in  performance.  ” 

• “Not  everyone  is  good  in  taking  tests.  Also  exams  and  homework  do  not  provide  the 
opportunity  through  performance  tasks.  ” 

• “Two  is  enough.  ” 
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12.  Would  you  recommend  that  other  faculty  members  get  the  opportunity  to  learn  to 
develop  and  use  performance  tasks  and  rubrics  as  student  assessment  tools? 

(7)  Yes  Not  really 

Why? 

• “1  believe  this  was  one  of  the  most  beneficial  aspects  of  the  program  with  regard  to 
student  learning  and  assessment.  It  ties  in  with  active  learning  and  Bloom ’s 
Taxonomy.  ” 

• “It  was  a big  help  for  me.” 

13.  Was  the  performance/rubric  development  process  used  with  this  group  - “developing 
while  learning  from  presentation,  examples,  and  one-on-one  feedback”  - effective? 

(7)  Yes  Not  really 

Why? 

• “ One-on-one  feedback  especially  helpful.  ” 

14.  Will  you  continue  to  use  performance  tasks  and  rubrics  in  this  and/or  other  classes? 

(7)  Yes  Not  really 

Why? 

• “To  improve  student  learning.  ” 

15.  Strengths  of  the  performance  task/rubric  program  component. 

• “Already  stated  in  above  [ responses].  ” 

• “Measures  what  students  can  really  perform  with  their  learned  tools.  ” 

• “Quantified  student  performance;  gave  students  guidance  and  goals.  ” 

• “I  liked  the  development  of  the  Performance  Tasks,  especially  with  the  rubric. 
Discussions  were  enlightening,  as  well  as  our  group  discussions  and  evaluations.  ” 

• “Very  good  way  in  presenting  material;  Different  styles  of  rubrics  presented;  also 
working  in  our  same  classes  helped  to  learn  how  to  do  rubrics  and  performances.  ” 

• “Provide  other  teaching  styles,  learning  styles,  and  teaching  models.  ” 

• “Allow  for  active  learning.  Results  show  improvement  when  Performance  Tasks  were 
done  in  groups.  ” 

16.  Areas  to  improve  in  the  performance  task/rubric  program  component. 

• “Revisit  and  revise.  ” 

• “None.  ” 

• “Streamline  the  time  scale.  ” 

• “Good  as  is.  ” 

17.  General  comments: 

• “This  part  was  exceptional — I will  always  use  this  info  in  my  classes  in  the  future.  ” 

• “Very  good  program.” 

“Results  indicate  conclusively  that  learning  level  was  enhanced  [by  students  in  experimental 
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DISCUSSION  OF  PROFESSORS’  CONTENT  KNOWLEDGE  ON 
EDUCATIONAL  RESEARCH 

(see  Program  A. 5 and  Instruments  in  C.2  - Gilmer) 

Jerry  Gilmer,  Ph.D 

A knowledge  test  covering  research  topics  was  administered  to  the  professors  three  times: 
immediately  prior  to  the  research  teaching  sessions  (5/25/06),  the  same  day  immediately 
after  the  research  sessions  (5/25/06),  and  at  the  end  of  the  semester  during  which  the 
professors  were  practicing  the  concepts  learned  (12/15/06).  The  maximum  number  of  points 
from  the  test  is  14.  The  table  below  contains  the  professors’  scores  for  each  administration 
of  the  test. 


Table  B.2.f.l 

Admin  1 

Admin  2 

Admin  3 

(5/25/06) 

(5/25/06) 

(12/15/06) 

Abdel-Motaleb 

0 

9 

8 

Azad 

2 

5 

3 

Coller 

7 

11 

11 

Gupta 

5 

14 

9 

Mo  rag  a 

8 

9 

6 

Rahn 

- 

- 

11 

Tatara 

5 

11 

7 

Mean  = 

4.5 

9.8 

7.9 

SD  = 

3.0 

3.0 

2.9 

The  differences  between  the  means  across  the  professors  from  the  different  administration 
times  were  analyzed  statistically  and  the  results  are  presented  in  the  table  below.  The 
significance  levels  are  from  paired  samples  t-tests  on  the  data  from  six  professors  (df  = 5). 


Table  B.2.T2 


Mean 

SD 

Difference  From 
Admin  1 

Sig.  Levei 

Admin  1 
(5/25/06) 

4.5 

3.0 

- 

- 

Admin  2 
(5/25/06) 

9.8 

3.0 

5.3 

.010 

Admin  3 
(12/15/06) 

7.3 

2.7 

2.8 

.095 

The  data  in  the  table  indicate  that  the  increase  in  the  professors’  knowledge  on 
research  topics  from  the  first  administration  to  the  second  was  statistically  significant 
beyond  the  .05  level  of  significance.  However,  the  professors’  knowledge  tended  to 
decline  slightly  during  the  semester  in  which  they  were  applying  the  knowledge  in 
their  own  classrooms,  rendering  the  difference  between  Admin  1 and  Admin  3 not 
statistically  significant. 
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CEET  FACULTY  DEVELOPMENT  PROGRAM 
FEEDBACK  AND  EVALUATION  FROM  PARTICIPATING  PROFESSORS 

(see  Research  in  A. 3,  Program  in  A. 5 and  Instruments  in  C.2  - Scarborough) 

Jule  Dee  Scarborough,  Ph.D. 

The  CITL  Faculty  Development  Program  was  extremely  demanding,  time  consuming,  and  work 
oriented.  The  faculty  members  were  expected  to  learn,  analyze,  perform,  and  produce  educational 
products  as  well  as  make  instructional  decisions  about  educational  processes,  models  and  techniques 
as  they  participated.  The  program  was  interactive  and  integrated,  actively  engaging  the  professors 
while  learning.  To  perform  to  the  level  that  this  group  did  exhibited  great  commitment  to  teaching 
and  learning,  both  for  themselves  and  their  students.  It  also  clearly  showed  commitment  to  their 
respective  departments  and  the  college,  and  especially  to  each  other  as  colleagues.  Their  feedback 
was  positive,  constructive,  and  well-intentioned.  Their  comments  on  each  feedback  form  reveal 
suggestions  that  are  worth  considering  for  the  next  stage  of  the  program.  Very  important  to  note  is  the 
respect,  friendly  manner,  and  consideration  that  they  showed  to  the  program  leaders.  They  were 
colleagues  and  professionals  in  every  sense.  (Note:  See  individual  feedback  summaries  that  follow.) 

Program  feedback  and  evaluation  was  a variable  in  the  research  and  evaluation  design  to 
determine  program  success.  The  professors  provided  feedback  after  each  program  segment. 
For  some  segments,  that  was  one  day;  however,  for  other  segments,  the  number  of  days  could 
vary  from  three  to  five.  The  only  segment  where  the  feedback  was  not  collected  at  the 
immediate  end  was  the  performance  assessment  component.  However,  they  completed  that  at 
the  January  2007  meeting,  which  turned  out  to  be  more  beneficial,  as  the  results  of  the 
feedback  incorporated  how  professors’  felt  about  their  performance  tasks  and  rubrics  after 
they  were  used  with  the  students.  Although  a very  great  majority  of  the  responses  to  items 
and  the  comments  were  very  positive,  there  were  comments  or  suggestions  to  consider  for 
improving  the  program  or  professors’  willingness  to  participate. 

Topics  of  Interest  for  Further  Study 

1 . More  on  Educational  Research  - beyond  superficial  to  engage  in  more  depth 

2.  Additional  time  on  Cooperative  Learning  - deeper  level 

3.  Additional  topic  on  Student  Teaming  - student  team  development,  team  work,  and 
grading 

4.  Program  component  on  Conflict  Management  for  Students  - civility  in  the  classroom; 
classroom  management 

A summary  of  each  feedback  form,  with  all  comments  and  notations,  follows  on  the  chart 
below  for  the  entire  program.  However,  to  review  each  individual  program  component 
summary,  see  pages: 

1-12  for  the  end-of-research  semester  and  final  program  feedback  and  evaluation 
(Dec.06) 

13-16  for  the  end  of  program  feedback  and  evaluation  (May,  2006) 

17-20  for  the  Performance  Assessment  feedback  (Jan.  2007) 

See  pages  21-30  for  the  feedback  summaries  on  each  individual  program  component.  All  are 
summarized  on  the  chart  in  Table  1 below. 


1 


Table  B.3.1:  Summary  - February  2006-  January  2007 


Prompted 

Responses 

Very 

Positive 

Constructive  Comments: 

Usually,  these  comments  were  made  by  the  same  one 
individual  throughout  the  feedback  process. 

Comments 
Very 
Positive 
with  few 
exceptions 

Orientation 

(2/2/06) 

X 

Lots  of  definitions  that  do  not  mean  much.  [This  seems 
to  have  changed  as  a result  of  the  program.] 

X 

Course  Analysis 
(2/9/06) 

X 

No  considerations 

X 

Test  Analysis 
(2/16/06) 

X 

No  considerations 

X 

Student  Learning 
Outcomes 
(3/2/06  & 3/23/06) 

X 

No  considerations 

X 

Test  Development 
(3/30/06  - 4/27/06) 

X 

No  considerations 

X 

Performance 
Assessment  & 
Rubrics  (12/06) 

X 

No  considerations 

X 

The  Big  Picture.  Would  be  useful  if  the  program  was 
laid  out  at  the  beginning. 

[We  really  did  this  but  believe  because  the  words  and 
educational  terminology  had  little  meaning  to  the 
participants,  full  understanding  of  what  was  explained  did 
not  occur.  This  can  be  modified  to  assist  better 
understanding.  Also  because  the  program  leaders  had  no 
idea  what  would  be  required  to  move  the  group  toward 
achievement  of  program  goals,  the  exact  steps  could  not 
be  identified  until  tried  and  adjusted  to  fit  group.] 

End  of  Program 
(5/15/06  - 5/25/06) 

X 

One  individual  commented  on  the  avalanche  of 
learning  materials  and  suggested  to  start  small  and 
then  expand  to  reduce  confusion  and  being 
overwhelmed. 

[It  is  true,  they  lefi  the  program  with  a complete  toolbox  of 
books,  articles,  and  other  materials,  ( e.g.  many  rubric 
examples,  etc.).  We  assumed  they  would  be  more  likely  to 
continue  if  they  had  the  resources  at  their  fingertips.  So  in 
trying  to  provide  a complete  toolbox,  we  knew  it  would  be 
somewhat  overwhelming,  and  it  was.  However,  we  can 
suggest  improvements  for  managing  so  many  different 
resources  and  materials  for  the  next  group.] 

X 

Incorporate  small,  interlinked  lectures  over  a period  of 
workshops  with  examples. 

[This  is  really  what  we  did,  so  do  not  know  how  to 
change  this.] 
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Make  a series  of  small,  interlinked  workshops  over 
time. 

\We  did  this  the  first  half  of  program  - 9 days  beginning 
in  February  and  ending  in  April,  but  then  finished  the 
program  with  two  whole  weeks  of  consecutive  days  in 
May,  so  professors  could  finish  during  intersession.  The 
two  weeks  were  very  productive,  but  very  intense  and 
focused.  The  quality  of  work  was  excellent  for  both 
schedules,  one  day  at  a time  and  the  block  of  consecutive 
days.] 

End  of  Program 
After  Research 
Semester 
(12/15/2006  and 
1/30/2007) 

X 

Program  time  commitment  was  mentioned  related  to 
the  number  of  days,  hours,  and  hours  per  day. 

[We  agree  that  18  days  is  probably  unheard  of  nationally, 
but  the  program  was  intentionally  designed  for  breadth 
and  some  depth.  One  of  the  most  important  factors  on  time 
in  the  program  was  that  time  was  allowed  for  professors  to 
actually  design,  develop,  and  produce  the  educational 
products  for  their  courses,  make  instruction  choices  about 
new  models  and  processes,  and  plan  how  to  use  them,  etc. 
This  takes  time.  This  was  not  a”  talk  and  walk”  program, 
leaving  the  follow-through  to  the  professors.  It  carried 
them  from  analysis  to  experimental  research  in  their 
classroom,  analysis  after  that  semester,  and  manuscript 
production.^  One  individual  felt  that  faculty  should 
have  time  off  to  take  the  course.  [These  are  interesting 
comments,  as  the  Dean  allowed  us  to  use  “in-semester” 
work  days  for  half  the  program,  even  though  faculty 
members  were  paid  a stipend  of  $5000.  The  other  half  was 
summer  time.  We  felt  professors  were  financially  rewarded 
well,  as  teaching  is  fully  part  of  their  job  responsibility^ 

Compensation  level:  one  individual  thought  it  should 
he  higher. 

[Each  faculty  member  received  $5000;  we  felt  they  were 
well  rewarded,  plus  resources,  materials,  and  the  Dean 
permitted  use  of  semester  days  for  one  half  of  program, 
not  all  summer.] 

One  individual  commented  that  this  program  is  best 
for  faculty  members  with  very  positive  attitudes; 
otherwise,  one  or  two  with  more  negative  mindsets 
would  drag  the  rest  down. 

\We  provided  several  different  perspectives  about  students. 
One  professor  was  fairly  negative  about  students  and  their 
commitment  to  learning;  however,  generally,  I do  not 
believe  we  really  had  any  negative  faculty  members.  We 
had  strong  personalities,  differences  of  opinions,  and  a 
variety  of  philosophies  about  teaching  and  students.  Thus 
the  group  was  rather  diverse  in  many  ways,  ethnicity, 
cultures,  disciplines,  one  female  participant,  the  principal 
leader  was  female,  but  no  one  in  the  group  was  negative.] 

X 
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It  is  possible  that  one  or  more  very  negative  individual(s) 
could  have  diverted  program  time;  however,  when  those 
individuals  are  included  and  ultimately  buy  in,  they  are  the 
best  advertisement  and  motivators  towards  sustainability. 
So  we  agree,  but  would  not  have  chosen  only  positive 
individuals  intentionally;  however,  the  Dean  did  select 
individuals  he  thought  would  be  willing  to  commit  to  such 
an  extended  program. 

One  individual  commented  that  maybe  workshops  in 
the  program  should  be  given  out  of  town  to  reduce 
interference  with  other  normal  job  tasks. 

[There  were  very  few  interruptions;  some  at  first  while 
getting  used  to  the  program,  process,  work  pace,  lab,  and 
delivery  schedule;  for  example,  one  professor  would  be 
interrupted  by  his  research  assistant  on  the  cell  phone.  But 
once  the  workshop  operational  standards  were  made  clear 
- no  interruptions  unless  absolutely  necessary,  our 
professors  showed  up  on  time  and  worked  hard  until  the 
day  ended  with  very  few  interruptions.  We  tried  to 
accommodate  various  lunch  time  preferences,  as  there 
were  prayer  time  preferences  and  family  lunch  times  to 
consider;  this  did  not  present  any  problems.  If  anyone  did 
not  quite  finish  the  day’s  work,  they  were  expected  to  be 
ready  for  the  next  session  with  work  in  hand.  That  did  not 
always  happen;  one  felt  that  he  was  not  getting  paid  for 
‘out-of-class  ’ work,  but  overall  the  pace  among  professors 
worked  for  the  most  part.  That  one  individual  did  not 
really  understand  that  teaching  IS  part  of  his  job  from  the 
perspective  that  any  time  preparing  for  class  - the  whole 
program  - was  his  responsibility.  This  individual 
performed  well. 

To  end  with  a professor’s  quote: 

“It  [Teaching  and  Learning  - research  on  TL]  is  a 
fantastic  area,  rich  with  opportunities  for 
grants/research  papers.” 
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CEET  Initiative  on  Teaching  and  Learning  (7/7  present) 

Einal  Program  Eeedback  - December  15, 2006 

1.  Looking  back,  and  after  the  research  semester,  do  you  feel  that  the  faculty  development  sessions  were 
worth  your  time  - worthwhile? 

-Course  Analysis:  e.g.,  content  gap  analysis  & priority,  learning  styles,  teaching  models 
& styles,  standards  & learning  objectives/outcomes,  objectives  and  test  item  match. 


Bloom’s  Taxonomy,  Dale’s  Cone,  Critical  Thinking,  and  more 
Not  really 

Yes  (6) 

Majority  (1) 

-Test  Analysis  and  Test  Development  Review 
Not  really 

Yes  (6) 

Majority  (1) 

- Performance  Assessment  and  Rubies 
Not  really 

Yes  (6) 

Majority  (1) 

-Analyzing  all  your  assessments  by  Bloom 
Not  really 

Yes  (6) 

Majority  (1) 

- Consideration  of  “broader”  assessments  and  mapping  your  assessments 
Not  really 

Yes  (6) 

Majority  (1) 

-Teaching  Models,  including  Cooperative  Learning  and  Mapping  Analysis 
Not  really 

Yes  (6) 

Majority  (1) 

-The  review  of  components  for  a more  revealing  syllabus  for  the  students 
Not  really 

Yes  (6) 

Majority  (1) 

-The  review  and  consideration  of  Multicultural  aspects  of  courses. 
Not  really)  1) 

Yes  (3) 

Majority  (2) 

-The  review  and  consideration  of  grading. 
Not  really 

Yes  (3) 

Majority  (4) 

-The  educational  Research  Session 
Not  really 

Yes  (4) 

Majority  (3) 
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-Overall,  was  the  faculty  development  program  worth  your  time  & “worthwhile”?  Yes  (6)  Majority(l) 
Not  really 

****Why  or  Why  not  to  any  of  the  above  statements? 

• “I  really  liked  the  content.  However,  it  was  an  enormous  time  commitment,  which  made  it  very 
difficult” 

• “The  addition  of  rubrics  had  a tremendously  positive  impact” 

• “Gained  an  appreciation  for  educational  research  and  the  power  it  holds” 

• “Test  analysis  was  very  useful;  I will  continue  to  use  it.  ” 

• “Wish  we  had  more  on  cooperative  learning/team  dynamics” 

• “This  was  the  best  faculty  development  program  that  I had  participated  in  at  NIU.  The  program 
addressed  all  aspects  of  a course  development,  course  analysis,  test  analysis,  test  development, 
and  modifying  a course  in  an  organized  manner.  Also  using  assessment  as  a process  for 
continuous  improvement.  ” 

• “It  was  worth  the  time  & ‘worthwhile  ’ because  it: 

(i)  introduced  me  to  various  concepts,  styles,  etc.  that  are  typically  considered  only  by  people  in 

colleges  of  education 

(ii)  introduced  new  dimensions  to  the  course  by  considering  broader  assessments  as  well  as 
evaluation  of  assessment  (rubrics) 

(Hi)  improved  learning  of  students  thru  better  syllabus  (content  and  relationship  to  SLO  -student 
learning  objectives),  etc.  ” 

• “More  Research  sessions  would  have  been  desirable  (I  want  to  learn  more)” 

• “More  time  on  topics  like  Multicultural  aspects.  Applications  of  Teaching  Models,  Cooperative 
Learning,  etc.  would  be  recommendable  ” 

• “I  personally  missed  the  topic  of  keeping  civility  in  the  classroom  or  throughout  the  semester.  ” 

• “This  course  exposed  me  to  new  types  of  knowledge;  I could  not  know  this  if  I did  not  take  this 
course  or  it  is  unlikely  to  know  this  knowledge  on  my  own.  ” 

2.  Would  you  recommend  the  overall  program,  including  the  program  content  as  you  experienced  it,  for 
other  faculty  members? 

Yes,  definitely  (3)  Yes,  the  greater  majority  of  it  (4)  Some  of  it  Not  really 

****Why  or  why  not? 

• “Only  for  faculty  members  with  very  positive  attitudes  and  open  minds.  Negativity,  even  among 
just  a few  members,  can  drag  the  rest  down.  I think  a less  ambitious  workshop  might  be  more 
effective  because  participants  won’t  feel  as  constrained  by  time.  ” 

• “Yes,  I strongly  believe  that  other  faculty  members  should  be  exposed  to  this  kind  of  program. 
This  kind  of  activity  will  improve  the  overall  teaching  quality  of  CELT.  ” 

• “It  would  help  faculty  members  to  improve  their  courses,  which  would  eventually  lead  to  better 
prepared  students.  ” 

• “I  think  the  program  is  good  for  instructors  [professors]  like  us,  who  don’t  have  background  in 
these  matters  ” 

• “Setting  needs  to  be  different.  May  be  the  course  should  be  given  in  a workshop  form  out  of  town 
to  reduce  interfering  with  other  normal  job  tasks  ” 
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3.  Looking  back  at  the  semester,  were  the  modifications  made  to  your  course  or  the  changes  to  your 
course  worthwhile  and  effective  with  students? 


Yes,  definitely-they  improved  the  course  and  instruction  (3) 
Yes,  some  of  them  improved  the  course  and  instruction  (1) 


Yes,  the  majority  of  them  improved  the 
course  and  instruction  (2) 


Not  as  many  of  the  changes  were  as  valuable  as  I had  hoped  (1)  No,  very  few  of  them  were  successful 

changes 


****Why  or  Why  not? 

• “Every  course  is  different.  Hence  the  course  materials  should  be  customized  for  the  individual 
course.  But  this  course  should  be  given  to  professors  as  is  because  it  exposes  them  to  all 
techniques  ” 

• “I  think  the  success  of  these  methods  depends  also  on  students.  That  component  is  critical  - 
orientation  for  students  on  these  issues  would  be  great” 

• “Through  improved  syllabus,  performance  assessments,  etc.  ” 

• “In  terms  of  the  students  for  the  target  course,  it  was  too  many  changes  within  one  semester. 
Considering  the  course  delivery  methods  for  [???]  program,  this  course  delivery  was  a major 
shift.  So  it  was  little  difficult  for  the  student  to  grasp  all  the  benefits" 

• “Much  better  organized;  much  better  assignments.  Much  better  tests.  ” 

• “The  tight  deadlines  for  course  content  do  not  always  work,  especially  in  a senior  level  course 
and  especially  when  students  bring  in  new  topics,  ideas  that  they  would  like  to  discuss  ” 

4.  Were  the  teaching  and  learning  materials  developed  during  the  faculty  development  sessions  effective 
when  used  with  students  during  the  research  semester? 

Yes,  definitely  (1)  Yes,  greater  the  majority  of  them  (5)  Yes,  some  of  them  (1) 

Not  as  many  of  the  materials  were  as  valuable  as  I had  hoped  No,  very  few  of  the  materials  were 

successful 


****Why  or  Why  not? 

• “See  above"  [“Every  course  is  different.  Hence  the  course  materials  should  be  customized  for  the 
individual  course.  But  this  course  should  be  given  to  professors  as  is  because  it  exposes  them  to 
all  techniques  ”] 

• “I  had  to  design  additional  assignments  for  my  class” 

• “The  materials  developed  were  very  much  useful,  as  I have  used  them  for  the  course  without 
spending  much  time  during  the  semester” 

• “The  rubrics  for  the  performance  tasks  were  excellent” 

• “The  overall  course  organization  was  also  beneficial” 
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5.  Describe  how  you  feel  about  the  “products”  you  have  developed,  their  purpose,  usefulness,  quality, 
etc.? 


Analysis  products  - gaps  analysis,  teaching  models  and  styles  analysis,  learning  styles  analysis,  course  content 
Analysis 

• “/  guess  it  was  nice  to  see,  but  I think  I knew  instinctively  what  it  was" 

• “Very  Good” 

• “Useful  exercise” 

• “Useful” 


Syllabus  - 

• “This  was  good,  although  it  didn’t  have  as  big  of  an  impact  as  other  aspects” 

• “V.  good” 

• “It  was  good” 

• “Very  Good” 

• “Essential  product” 

• “Useful” 

• “Very  useful  - 1 do  not  like  the  tight  deadlines  though" 


Tests  - 

• “I  really  appreciate  the  diagnostic  info  I get  from  my  revised  test(s)” 

• “Excellent” 

• “Good” 

• “ Convenient  for  test  preparation” 

• “Useful” 

• “The  multiple  choice  t/f  questions  did  not  seem  to  work  well I will  keep  my  “short  answer” 
type” 


Test  Analyses  - 

• “I  really  appreciate  the  diagnostic  info  I get  from  my  revised  test(s)” 

• “Good” 

• “This  was  very  helpful  to  me  ” 

• “Very  Good” 

• “Provides  understanding  about  a test” 

• “Useful” 

• “Very  beneficial” 

Performance  Assessments  - 

• “ PA#2  and  PA#3  were  the  best  learning  experiences,  I think,  for  the  student’s  high  impact” 

• “Very  Good” 

• “Was  good” 

• “Very  Good” 

• “Useful” 

• “Excellent!” 


Rubrics  - 

• “/  had  rubrics  before.  Now  they’re  a bit  better” 

• “V.  good” 

• “Was  good” 

• “Very  Good” 

• “Convenient  for  scoring” 

• “Excellent  value  ” 

• “Excellent!  ” 
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Choices  of  teaching  models  and  processes  - 

• “Mixing  it  up  was  good.  ” 

• “Good” 

• “I  would  like  to  continue  using  collaborative  learning  styles” 

• “Good” 

• “ Most  were  useful” 

• “Very  useful  I liked  using  the  KOLB  inventory  with  the  students;  they  had  a good  response  to 
this  activity” 

6.  Was  the  Research  Semester,  performing  experimental  classroom  research  with  students,  worth  your 
time  and  “worthwhile”? 

• “yes  ” 

• “yes,  very  valuable  ” 

Yes,  very  valuable,  beneficial  in  the  following  ways: 

(5)  provided  evidence  of  the  benefit  of  the  course,  teaching,  learning,  content,  test,  etc.  gaps  analyses  and 

what  was  learned  from  them  and  developed  as  a result  of  identifying  the  gaps  through  the  analysis  processes 

(7)  provided  opportunity  to  evaluate  and  see  benefits  of  new  teaching  and  learning  or  educational  products 

(7)  provided  opportunity  to  evaluate  and  see  benefits  of  new  teaching  and  learning  processes 

(6)  provided  opportunity  to  evaluate  and  see  benefits  of  new  teaching  models  and  styles 

• “very  valuable” 

(7)  provided  insight  into  student  learning 

(7)  provided  insight  about  my  teaching 

• “let  me  know  that  some  of  what  I was  doing  had  a name;  opened  my  eyes  to  new  ideas” 

(6)  provided  opportunity  for  a first  attempt  at  educational  research  - scholarship  of  teaching 

• “this  is  invaluable  - 1 will  definitely  pursue  this  ” 

(1)  identify  and  list  others; 

• “interesting,  fun  ” 

Not  really  as  valuable  as  I had  hoped:  describe  why  for  each  item  below: 

provided  evidence  of  the  benefit  of  the  course,  teaching,  learning,  content,  test,  etc.  gaps  analyses  and 

what  was  learned  from  them  and  developed  as  a result  of  identifying  the  gaps  through  the  analysis 
processes 

provided  opportunity  to  evaluate  and  see  benefits  of  new  teaching  and  learning  or  educational  products 

provided  opportunity  to  evaluate  and  see  benefits  of  new  teaching  and  learning  processes 

provided  opportunity  to  evaluate  and  see  benefits  of  new  teaching  models  and  styles 

provided  insight  into  student  learning 

provided  insight  about  my  teaching 

provided  opportunity  for  a first  attempt  at  educational  research  - scholarship  of  teaching 
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(2)  identify  and  list  others: 

• “Cooperative  learning,  team,  leadership  I would  like  to  study  these  areas  more  (need  more 
time)’’ 

• “Looking  at  the  data  now,  I wish  I had  designed  some  parts  of  the  experiment  better.  ” 

1.  Would  you  recommend  to  other  faculty  members  that  they  begin  to  engage  in  research  on  teaching 
and  learning? 

Yes,  definitely  (7)  Yes,  definitely,  but  after  participating  in  the  faculty  development  to  prepare  them 

No,  not  really 

*****Why?  Specifically,  what  would  keep  you  from  recommending  that  others  engage  in  classroom 
research  on  teaching  and  learning?  Please  describe  in  detail. 

• “Because  it  provides  a source  of  information  that  rarely  is  available  to  people  in  profession  other 
than  teachers.  ” 

• “I  think  this  is  a very  good  policy  of  the  college  ” 

• “This  will  improve  overall  quality  of  teaching  for  our  CELT.  ” 

8.  From  the  perspective  of  the  entire  program  (faculty  development,  the  development  of  course  and 
classroom  materials,  and  the  educational  research  semester)  was  the  entire  program  - beginning  with 
analysis  through  classroom  research  as  a “whole”  program,  worth  your  time  - worthwhile? 

Yes  (6)  The  greater  majority  (1)  Some  of  it  Really,  Not  much 

****Specifically,  Why  or  Why  not? 

• “It  is  a unique  chance  to  learn  this  type  of  knowledge.  ” 

• “Even  though  I have  taught  this  course  many  time  before,  it  allowed  me  to  evaluate  the  course 
from  a completely  new  perspective  and  improve  upon  it” 

• “I  now  have  a better  understanding  about  how  students  learn  and  how  I can  improve  my  courses 
through  properly  organized  teaching.  ” 

• “It  was  such  a big  time  commitment,  that  I can ’t  say  yes  completely.  ” 

• “It  is  important  to  take  our  jobs  as  educators  seriously 

• “We  should  strive  for  continuous  process  improvement/lifelong  learning  ourselves" 

• “It  is  a fantastic  area  rich  with  opportunities  for  grants/research/papers” 

9.  Would  you  recommend  that  other  faculty  members  get  the  opportunity  to  participate  in  this  “whole” 
program,  including  faculty  development,  course  development,  and  classroom  research  ? 

Yes,  definitely  (4)  Yes,  with  a few  content  changes  (3)  Yes,  with  many  content  changes? 

No,  not  really 

10.  Specifically,  what  content  changes  would  you  suggest? 

• “More  cooperative  learning  ” 

• “Less  time  on  the  syllabus  timeline  ” 

• “As  I said  before,  I think  a less  ambitious  approach  might  be  more  effective.  E.g.  Just  one 
performance  assignments. ..I’m  not  sure  where  to  cut.  Syllabus?” 

• “Exposed  them  slowly  to  program  ” 

• “Introduce  aspects  of  civility  in  class  room.  ” 

• “How  to  manage  conflicts.  ” 

• “See  above” 
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11.  Was  the  learning  and  development  “process”  used  during  the  entire  program  (Oct.05-research, 
Dec.06)  effective? 

Yes,  definitely  (5)  Yes,  with  a few  process  changes  (1)  Yes,  with  many  process  changes 

No,  not  really  No  response  (1) 

****-\yj^y7 

• “I  think  I learned  certain  important  issues  on  education  that  will  help  me  to  improve.  ” 

• “Covered  a lot  of  material  with  hands-on  experience” 

12.  Specifically,  what  process  changes  would  you  suggest?  ****Why? 

• “Have  workshop  days  not  be  always  so  work  intensive  - these  were  very  draining  and  it  took  time 
to  recover” 

• “I  sincerely  think  the  way  instructors  file  materials  from  us  should  change.- 1 feel  like  they  ask  us 
too  many  times  the  same  information  ” 

• “Faculty  should  have  time  off  to  take  the  course.  This  is  a huge  task  that  cannot  be  added  to  the 
heavy  load  of  faculty.  ” 

13.  Specifically,  what  about  the  program,  overall,  would  keep  you  from  recommending  it  to  other  faculty 
members? 

Describe  in  great  detail. 

• “The  time  commitment  - it  was  very  intense  ” 

• “Faculty  should  know  the  complete  program  plan  at  the  very  beginning” 

• “It  should  be  recommended  to  any  faculty  who  is  very  committed  to  put  lot  of  time  and  energy, 
open  to  new  ideas  and  concepts  and  eventually  improve  the  course” 

• “ -^Everything  was  good” 

• “All” 
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14.  Identify  and/or  describe  the  teaching  and  learning  changes  that  you  implemented  in  the  research 
semester’s  course: 


(6)  Improved  priority  of  course  content 

(7)  New  syllabus  with  many  new  components 

(7)  Clear  learning  objectives/outcomes  tied  to  ABET/NAIT  standards 

• “Very  Useful!!!” 

(7)  Learning  Style  Inventory  (e.g.  Kolb,  Felder,  other) 

(7)  New  teaching  models 

- (4)  Small  groups 

• “mixing  inductive/deductive  ” 

• “included  more  group  discussion” 

• “Included  more  group  discussion  and  small  group  encounters” 

- One  minute  papers 

(7)  New  teaching  styles 

(7)  New  objective  tests 

(7)  New  performance  assessments/rubrics 

(6)  New  grading  criteria  - clear  and  pre-determined,  no  curving  of  grades,  or  last  minute  non-criteria- 

based  Judgments,  etc. 

(6)  Better  alignment  of  syllabus,  teaching,  and  assessment. 

Identify  and  list  others  specific  to  you. 

15.  Do  you  feel  that  the  program  dates  worked... recall  that  some  time  was  spent  during  the  regular 
semesters  and  some  time  during  the  summer;  specifically  there  was  V2  or  9 days  across  the  regular 
semesters  and  V2  or  9 days  in  May,  plus  2 fall  meetings,  and  the  final  meeting? 

Worked  well  (5)  Would  prefer  a different  schedule  (1)  No  response  (1) 

****Describe  a preferred  schedule  for  18  days,  plus  several  short  meetings 

• “Should  be  given  during  breaks” 

• “ Just  a bit  overwhelming  ” 

16.  Describe  the  strengths  of  the  overall  program  content. 

• “Everything  is  good” 

• “Help  faculty  who  are  not  familiar  with  educational  research” 

• “Covers  all  aspects  of  teaching,  assessment,  evaluation,  and  course  development” 

• “I  loved  doing  the  research  ” 

• “Excellent  exposure  to  the  whole  idea  of  the  scholarship  of  teaching” 

• “Many  excellent  ideas  for  course  improvement” 

• “Introduce  “civility”  and  “conflict’  issues  in  [program]  contents” 
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17.  Describe  the  strengths  of  the  overall  learning  and  development  process. 

• “Results  in  better  learning  for  students” 

• “Definitely,  I think  this  is  a good  program  for  the  college!” 

• “Teaching  and  Learning  Style” 

• “ Constructivism  ” 

18.  Describe  the  areas  you  would  like  to  see  improvements  in  regarding  the  overall  program  content 
AND  specify  the  desired  improvements. 

• “None  that  I can  think  of.  Only  suggestion  — that  faculty  should  be  allowed  to  adjust  the  grading 
cut  offs  based  on  difficulty  of  questions  in  tests.  ” 

• “[Add]  Coop  Learning,  Team  Concepts,  Leadership 

• “Already  stated” 

• “Timing  - load  management.  Compensation.” 

19.  Describe  the  areas  to  you  would  like  to  see  improvements  in  related  to  the  overall  learning  and 
development  process  AND  specify  the  desired  improvements. 

• “It  was  very  good.  ” 

• “Ditto  ” 

20.  Dean  Vohra  would  like  your  Learning  Community  to  continue  and  actively  involve  each  of  you 
together  to  continue  to  learn,  share,  and  execute  research  on  teaching  and  learning.  At  this  point, 
although  we  don’t  have  it  well  defined,  are  you  willing  to  help  define  what  “continued  action”  means  and 
then  continue  to  participate? 

Yes,  definitely  (7)  “Very  good  idea”  No,  probably  not  - Why? 

Describe  for  you. 

21.  Other  General  Comments  about  the  overall  program: 

• “Thank  You.  I will  retain  this  info  forever  and  expand  upon  it.  ” 

• “Thank you  very  much.  It  was  a wonderful  experience.” 
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CEET  Initiative  on  Teaching  and  Learning 
Program  Feedback-  Final  Feedback  - May  15-25  & Overall  (7/7  present) 

1.  Were  the  sessions  May  15-25  worth  your  time  - worthwhile? 


Test  Analysis  and  Development  Review 

(7) 

Yes 

Not  really 

Performance  Assessment  and  Rubies 

(7) 

Yes 

Not  really 

Analyzing  all  your  assessments  by  Bloom 

(7) 

Yes 

Not  really 

Consideration  of  “broader”  assessments  and  mapping  your 

(7) 

Yes 

Not  really 

assessments 

Teaching  Models,  including  Cooperative  Learning  and  Mapping 

(7) 

Yes 

Not  really 

Analysis 

The  review  of  components  for  a more  revealing  syllabus  for  the 

(7) 

Yes+(1)  Not  really 

students 

The  consideration  of  Multicultural  aspects  of  courses. 

(7) 

Yes 

Not  really 

The  consideration  of  grading. 

(7) 

The  Research  Session 

(6) 

Yes 

(l)Not  really 

Why? 

• “Most  of  the  information  provided  are/seems  to  be  useful.  ” 

• “I  think  we  should  assess  whether  it  can  be  simplified  at  all.  It  was  exhausting.  ” 

• “Some  like  grading  and  multicultural  issues  were  not  addressed  earlier.  Also  analyzing  my  course 
in  terms  of  Bloom,  teaching  models,  etc.  made  me  appreciate  various  techniques  of  learning.  ” 

• “I  think  the  Research  Session  was  really  good.  But,  I feel  the  Research  Session  should  have  had 
more  time  to  really  understand  what  we  are  going  to  do  in  our  class  in  fall.  ” 


2.  From  the  perspective  of  the  entire  program,  was  it  as  a “whole”  worth  your  time  - worthwhile? 

(7)  Yes  Not  really 


Why? 

• “It  was  a good  introduction  in  two  areas:  a)  How  to  plan,  analyze,  and  update  a course  as  a 
continuous  process  and  (b)  Basics  of  educational  research.  ” 

• “Only  if  they  are  sufficiently  open-minded  and  care/interested  enough  about  teaching/learning  to 
put  in  the  effort.  ” 

• “Hopefully  students  will  learn  more  due  to  better  analysis  and  delivery  of  course.  ” 

• “Opened  my  eyes  to  a whole  new  area-  definitely  improvements  can  be  made  in  the  classroom 
armed  with  this  new  knowledge.  ” 


3.  Would  you  recommend  the  overall  program  content  as  you  experienced  it  for  other  faculty  members? 


(6)  Yes  (1)  Not  really 


Why? 

• “Invaluable  ” 

• “I  prefer  to  try  some  of  these  techniques  in  one  of  my  classes  and  evaluate  their  impact.  This  will 
allow  me  to  have  a credible  standpoint.  ” 

• “See  above  ” 

• “Because  it  will  improve  the  learning  of  students.  However,  some  faculty  may  not  be  open  to 
change.  ” 

• “It  seems  to  me  that  our  college  should  implement  this  workshop  as  a “must”  policy  for  every 
faculty.  For  that  way,  we’ll  make  sure  students  will  receive  a homogeneous  “product”  from  all 
[faculty?]” 

• “One  thing  I would  change  is  to  start  small  then  expand.  Providing  avalanche  of  materials 
caused  a lot  of  confusion  to  me.  May  be  start  with  one  learning  set  first.  ” 
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4.  Would  you  recommend  that  other  faculty  members  get  the  opportunity  to  participate  in  this  program 
with  content  modifications? 

(7)  Yes  Not  really  - too  soon  to  determine 

What  changes  would  you  suggest? 

• “Make  a series  of  small  and  interlinked  workshops  over  the  period  of  time.  This  will  allow  them 
to  digest  the  material  and  introduce  changes  in  a progressive  way.  ” 

• “A  bit  intense” 

• “However,  it  should  be  tried  out  first  half  next  semester  before  it  is  recommended  for  other 
faculty.  ” 

• “I  would  incorporate  more  lectures  in  middle  of  sessions  with  examples.  ” 

• “The  handouts  of  material  were  sometimes  overwhelming.  I would  reduce  this.  ” 

• “Less  time 

5.  Was  the  learning  and  development  “process”  used  during  the  entire  program  (including  the  May  15- 
25  time)  effective? 

(7)  Yes  Not  Really 

Why? 

• “The  program  was  intensive  and  structured  and  the  tutors  [program  leadersjhave  personal 
background  of  using  the  techniques  for  their  own  courses.  Personal  experience  of  the  tutor 
[program  leader [ made  the  program  credible  one.  ” 

• “Yes,  but  handouts  are  too  much  to  handle” 

6.  Describe  how  you  feel  about  the  “products”  you  have  developed,  their  purpose,  usefulness,  quality, 
etc.? 

• “I  feel  proud.  I am  amazed  at  the  amount  of  work  accomplished.  Quality  is  excellent,  usefulness 
is  invaluable  ” 

• “Excellent” 

• “I  think  my  products  in  this  workshop  are  of  good  quality,  for  me,  they  will  be  of  great  help  in  fall. 
I feel  I have  good  material” 

• “Very  good” 

• “They  are  the  result  of  a lot  of  effort” 

• “The  products  that  I have  developed  are  useful  items  towards  the  proposed  course.  However,  they 
need  to  be  revisited  before  they  can  be  used  for  the  class.  ” 

7.  Describe  teaching  and  learning  process  differences  that  you  will  implement  in  next  fall’s  course? 

• “Definitely  will  include  more  active  learning,  more  discussion  groups,  less  “lecture” 

• “Will  use  performance  tests,  collaborative  learning,  improved” 

• “I  will  use  more  active  learning,  but  now  I feel  I know  how  to  proceed  to  make  this  more 
productive.  I will  also  incorporate  collaborative  methods.  ” 

• “Use  various  teaching  styles,  models;  be  aware  of  Bloom’s  levels  [of  learning  [” 

• “Much  more  active  and  hands-on.  More  collaborative.  ” 

• “The  course  will  be  much  more  learner  and  assessment  centered.  ” 
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8.  Do  you  feel  that  the  program  dates  worked. ..meaning  some  semester  time  and  some  summer  time;  our 
time  was  Vi  days  (9)  in  semester  and  Vx  days  (9)  in  May. 

(7)  Worked  well  Would  prefer  a different  schedule. 

Describe  a preferred  schedule  for  18  days. 

• “It  was  long,  but  by  restructuring  the  course  the  # of  days  can  be  reduced.  ” 

• “For  me,  worked  well.  ” 

• “However,  shorter  days  would  be  preferred” 

• “Worked  well... the  distribution  was  good.  The  intensity  and  time  commitment  was  high.  ” 

• “ However,  last  two  weeks  was  very  intense.  More  days  could  be  used  during  the  semester  time.  ” 

9.  Strengths  of  the  program  overall. 

• “A  plethora  of  useful  information” 

• “Real  work  was  accomplished  as  evidenced  by  our  “products  ” 

• “Good  discussion  with  fellow  faculty  members.  ” 

• “Presented  teaching  and  learning  in  a scientific  way.  ” 

• “It  provides  a great  deal  of  information  and  knowledge  about  what  we  are  supposed  to  be  and 
how  we  are  supposed  to  act  as  educators  ” 

• “Good  understanding  of  education  principles” 

• “So  much  interesting  stuff  to  think  about” 

• “Experience  of  the  tutors  [program  leaders]” 

• “Program  structure  ” 

10.  Strengths  of  the  learning  and  development  process  overall. 

• “See  Above” 

• “I  think  it  worked  well,  but  I feel  - as  I commented  above,  the  material  was  handed  out  in  an 
overwhelming  manner-  (even  though  I know  the  reasons).  But  this  could  go  against  the  learning 
process.  ” 

• “ It  worked  for  me” 

• “Mixture  of  lecture,  group  work,  and  individual  work” 

11.  Areas  to  improve  the  overall  program. 

• “Time  needed  was  enormous  - needs  to  be  shortened” 

• “See  3” 

• “I  would  incorporate  a little  bit  more  lectures  (short)  on  some  topics  and  examples” 

• “Shorter  days.  Fewer  content  (e.g.  only  one  taxonomy  for  teaching  styles.  Higher  compensation 
considering  the  time  required” 

• “Mentioned  above  ” 

• “It  would  be  useful  if  all  the  steps  of  the  program  could  be  made  available  at  the  very  beginning. 
Such  as  course  analysis,  course  development,  educational  research,  and  subsequent  long-term 
plan.  In  that  case,  participants  would  have  much  better  commitments  ” 

12.  Areas  to  improve  the  learning  and  development  process  overall. 

• “/  would  incorporate  a little  bit  more  lectures  ( short)  on  some  topics  and  examples  ” 
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13.  Dean  Vohra  would  like  the  your  Learning  Community  to  continue  and  actively  involve  each  of  you 
together  to  continue  to  learn,  share,  and  execute  research  on  teaching  and  learning.  At  this  point, 
although  we  don’t  have  it  well  defined,  are  you  willing  to  help  define  what  “continued  action”  together 
mean  and  then  continue  to  participate? 

(7)  Yes  No,  probably  not 

• “Yes,  definitely” 

• “Maybe  it  will  be  a good  idea  to  let  the  community  grow  on  its  own.  While  administration  can 
play  the  role  of  a facilitator.  ” 

14.  Did  you  learn  or  enhance  “other”  types  of  skills  through  the  program  process  (e.g.  computer  or 
others)? 

• “No” 

• “No” 

• “Not  much  ” 

15.  General  Comments  about  the  overall  program; 

• “Excellent  - thank  you  for  allotting  me  this  opportunity!  ” 

• “It  was  a good  program  to  enhance  my  effectiveness  as  a teacher.  ” 

• “Excellent” 

• “My  overall  comments  are  that  this  was  a very  good  and  enriched  experience  and  definitely  will 
help  me  to  improve  my  teaching  as  well  as  professionally  and  personally.  ” 

• “Good.  However,  instead  of  focusing  on  terminologies,  focus  primarily  on  the  outcome  of  this 
program.  ” 

• “Mentioned  above  ” 
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CEET  Initiative  on  Teaching  and  Learning  - Performance  Assessment  Feedback,  Jan, 

2007 

(7/7  respondents) 

1.  Was  the  time  spent  developing  performance  tasks  worth  your  time  - worthwhile? 

(7)  Yes  Not  really 

Why? 

• “A  lot  of  work,  but  they  really  engage  students” 

• “Allowed  me  to  think  about  what  students  should  be  able  to  perform  after  completing  the 
course.  ” 

• “Invaluable.  ” “Although  I always  give  ‘projects’,  I was  naive  to  many  of  the  aspects  of  a 
true  performance  task.  ” 

• “Performance  tasks  made  students  (1)  solve  open  ended  problems;  (2)  work  in  groups; 
(3)  identify  problems  and  try  to  have  multiple  solutions  and  then  justify  the  solution.  ” 

2.  Was  the  time  spent  developing  rubrics  for  scoring  the  performance  tasks  worth  your  time  - 
worthwhile? 

(7)  Yes  Not  really 

Why? 

• “Students  knew  what  was  expected  of  them;  grading  was  a bit  easier;  fewer  students 
challenged  their  grades.  ” 

• “Allowed  me  to  set  expectations  from  the  PAs.  ” 

• “The  students  really  responded  well  to  them!  They  liked  knowing  the  expectations  for 
performance  tasks.  ” 

• “I  didn  ’t  have  this  experience  before.  ” 

• “Rubrics  helped  students  understand  what  is  expected  of  them  and  how  they  will  be 
graded.  ” 

3.  Would  you  recommend  the  performance  task  program  content  for  other  faculty  members? 

(7)  Yes  Not  really 

Why? 

• “Valuable” 

“It  will  be  a good  exercise  for  others.  ” 

• “I  think  any  new  faculty  should  be  exposed  to  this  experience.  ” 

• “I  feel  students  learned  a lot  because  of  the  performance  tasks.  ” 

4.  Would  you  recommend  the  rubric  program  content  for  other  faculty  members? 

(7)  Yes  Not  really 

Why? 

• “It  will  be  a good  exercise  for  others.  ” 

• “This,  I believe,  is  a necessity.  ” 
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5.  Were  the  performance  tasks  a beneficial  addition  to  the  student  assessment  plan  for  your 

course? 

(7)  Yes  Not  really 

Why? 

• “It  was  another  dimension  of  assessment.  Some  students  who  did  not  do  well  on 
tests. . . really  shined  in  performance.  ” 

“Provides  additional  form  of  assessment  method.  ” 

• “It  added  a new  dimension  of  student  assessment;  also  these  performances  tasks  involved 
various  learning  styles.  ” 

6.  Were  the  rubrics  beneficial  for  scoring  the  performance  tasks? 

(7)  Yes  Not  really 

Why? 

• “Students  knew  what  was  expected  of  them;  grading  was  a bit  easier;  fewer  students 
challenged  their  grades.  ” 

• “Makes  the  scoring  process  easier.  ” 

• “It  made  it  easier  for  me  — and  the  students  also  responded  well.  IT  is  necessary  to  have 
a procedure  mapped  out  for  them  to  understand  the  expectations  and  levels.  ” 

• “They  make  grading  progress  easier.  ” 

• “Otherwise,  it  would  be  very  subjective  or  arbitrary.  ” 

7.  Were  the  performance  tasks  an  effective  tool  for  enhancing  student  learning? 

(7)  Yes  Not  really 

Why? 

• “Bigger,  more  authentic  tasks.  ” 

• “Students  can  demonstrate  what  they  can  perform  after  completing  the  course.  ” 

• “(l)It  allowed  for  many  more  teaching  styles  to  be  incorporated  in  the  course;  (2)  more 
learning  styles  were  also  included;  (3)  a good  tool  for  group  work  as  well.  ” 

• “They  really  understand  expectations.  ” 

• “Students’  learning  involves  various  learning  and  teaching  style,  and  models  and 
performance  tasks  provided  these  opportunities.  ” 

8.  Were  the  performance  tasks  an  effective  tool  for  measuring  student  learning? 

(7)  Yes  Not  really 

Why? 

• “It  was  another  dimension  of  assessment.  Some  students  who  did  not  do  well  on 
tests. . . really  shined  in  performance.  ” 

• “Students  can  demonstrate  what  they  can  perform  after  completing  the  course.  ” 

• “(1)  It  demonstrated  their  abilities  to  communicate  effectively;  (2)  It  demonstrated  their 
abilities  to  synthesize,  apply,  and  evaluate  subject  matter  content.  ” 

• “Students’  learning  may  not  be  completely  assessed  by  only  exams  and  homeworks.  ” 
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9.  Were  the  rubrics  an  effective  tool  for  scoring  the  outcomes  of  student  performances  on  the 
tasks? 

(7)  Yes  Not  really 

Why? 

• “It  was  another  dimension  of  assessment.  Some  students  who  did  not  do  well  on 
tests. . . really  shined  in  performance.  ” 

• “See  previous  comments.  ” 

• “Rubrics  provide  the  details  of  expected  outcomes.  ” 

10.  Were  the  rubrics  effective  in  helping  students  to  understand  more  about  what  you  expected 
them  to  do  by  revealing  the  standards  and  scoring  mechanism  with  them  up  front? 

(7)  Yes  Not  really 

Why? 

• “It  was  another  dimension  of  assessment.  Some  students  who  did  not  do  well  on 
tests. . . really  shined  in  performance.  ” 

• “Students  know  the  expectations.  ” 

• “See  previous  comments.  ” 

• “They  know  what  is  expected  of  them.  ” 

11.  Do  you  feel  that  more  formalized  performance  tasks  and  rubrics  improve  the  opportunity 
for  students  to  provide  evidence  of  learning? 

(6)  Yes  (1)  Not  really 

Why? 

• “It  was  another  dimension  of  assessment.  Some  students  who  did  not  do  well  on 
tests. . . really  shined  in  performance.  ” 

• “Not  everyone  is  good  in  taking  tests.  Also  exams  and  homework  do  not  provide  the 
opportunity  through  performance  tasks.  ” 

• “Two  is  enough.  ” 

12.  Would  you  recommend  that  other  faculty  members  get  the  opportunity  to  learn  to  develop 
and  use  performance  tasks  and  rubrics  as  student  assessment  tools? 

(7)  Yes  Not  really 

Why? 

• “/  believe  this  was  one  of  the  most  beneficial  aspects  of  the  program  with  regard  to 
student  learning  and  assessment.  It  ties  in  with  active  learning  and  Bloom ’s  Taxonomy.  ” 

• “It  was  a big  help  for  me.” 
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13.  Was  the  performance/rubric  development  process  used  with  this  group  - “developing  while 
learning  from  presentation,  examples,  and  one-on-one  feedback”  - effective? 

(7)  Yes  Not  really 

Why? 

• “ One-on-one  feedback  especially  helpful.  ” 

14.  Will  you  continue  to  use  performance  tasks  and  rubrics  in  this  and/or  other  classes? 

(7)  Yes  Not  really 

Why? 

• “To  improve  student  learning.  ” 

15.  Strengths  of  the  performance  task/rubric  program  component. 

• “Already  stated  in  above  [ responses].  ” 

• “Measures  what  students  can  really  perform  with  their  learned  tools.  ” 

• “Quantified  student  performance;  gave  students  guidance  and  goals.  ” 

• “I  liked  the  development  of  the  Performance  Tasks,  especially  with  the  rubric. 
Discussions  were  enlightening,  as  well  as  our  group  discussions  and  evaluations.  ” 

• “Very  good  way  in  presenting  material;  Different  styles  of  rubrics  presented;  also 
working  in  our  same  classes  helped  to  learn  how  to  do  rubrics  and  performances.  ” 

• “Provide  other  teaching  styles,  learning  styles,  and  teaching  models.  ” 

• “Allow  for  active  learning.  Results  show  improvement  when  Performance  Tasks  were 
done  in  groups.  ” 

16.  Areas  to  improve  in  the  performance  task/rubric  program  component. 

• “Revisit  and  revise.  ” 

• “None.  ” 

• “Streamline  the  time  scale.  ” 

• “Good  as  is.  ” 

17.  General  comments: 

• “This  part  was  exceptional — I will  always  use  this  info  in  my  classes  in  the  future.  ” 

• “Very  good  program.  ” 

• “Results  indicate  conclusively  that  learning  level  was  enhanced  [by  students  in 
experimental  semester].  ” 
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(6  Present;  1 absent) 

CEET  Initiative  on  Teaching  and  Learning 
Item  Writing  and  Test  Development  - Days  6,  7, 8,  9 - March  30,  April  6,  20,  27 

1.  Were  these  four  days  worth  your  time  - worthwhile? 

5 Yes  2 No  Response  No,  not  really 

Why? 

• “Thought  a lot  about  testing  (productively)” 

• “Made  me  develop  the  tests  based  on  SLOs” 

2.  Did  the  presentation  and  handouts  provide  appropriate  information  to  guide  you  in  item  writing  and 
test  development  without  overwhelming  you  with  too  much  material? 

6 Yes  1 No  Response  No,  not  really 

Why? 

3.  Were  you  able,  during  the  four  sessions,  to  write  many  appropriate  and  valid  test  items  and  to 
assemble  what  you  would  consider  good  tests  or  tests  more  fully  developed  than  the  ones  you  were 
using  before  now? 

4 Yes  1 written  response  only  1 No  Response  No,  not  really 

Why? 

• “Not  nearly  enough.  It  takes  a long  time  to  draw  [figures]” 

• “Because  we  had  to  write  objective  test  items” 

4.  As  a result  of  these  four  sessions  do  you  now  feel  you  have  greater  ability  and  confidence  in  writing 
items  and  developing  tests? 

6 Yes  1 No  Response  No,  not  really 

Why? 

• “Exam/test  preparation  will  be  much  more  structured.  ” 

5.  Would  you  recommend  that  other  faculty  members  have  the  opportunity  to  learn  more  about  writing 
items  and  developing  tests  through  workshops  similar  to  these? 

4 Yes  2 No  Responses  1 No,  not  really 

Why? 

• “too  soon  to  use  until  we  finish  our  program  and  further  explore  its  meaning.  ” 

6.  Was  the  “process”  used  during  these  days  effective? 

5 Yes  1 No  Response  1 Written  Response  Only  No,  not  really 

Why? 

• ”I  came  up  with  a lot  more  item  ideas  today,  now  that  I had  to  put  test  together.  Perhaps  could 
infuse  some  of  today’s  content  earlier.  ” 
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7.  Strengths  of  this  aspect  of  the  program. 

• “Item  Bank  will  be  useful  for  the  coming  years.  ” 

• “Forces  faculty  to  analyze  the  course  content  and  tests  in  great  detail. 

• “Made  it  clear  how  you  can  make  a fair  test.  ” 

8.  Areas  to  improve  in  this  aspect  of  the  program. 


9.  General  Comments: 

• “Make  it  10-3  so  that  other  dept,  business  can  be  attended  before  and  after  workshop. 
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CEET  Initiative  on  Teaching  and  Learning 
Student  Learning  Outcomes  - Days  4 & 5,  March  2 & 23  (7/7  present) 

1.  Were  these  two  days  worth  your  time  - worthwhile? 

Yes  7/7  Not  really 

Why? 

• “Never  thought  so  deeply  about  outcomes.  To  be  honest,  I thought  more  about  other  courses  than 
this  one.  ” 

• “Really  magnified  the  usefulness  of  the  techniques.  ” 

• “Analyze  SLO  in  more  detail.  ” 

• “The  SLO  will  help  students  to  understand  what  they  are  going  to  learn.  ” 

2.  Would  you  recommend  the  program  content  on  Student  Learning  Outcomes  for  other  faculty 
members? 

Yes  5/7  Not  really  1/7  “Not  at  this  stage.”  Skipped  Question  1/7 

Why? 

• “Same”  (“Never  thought  so  deeply  about  outcomes.  To  be  honest,  I thought  more  about  other 
courses  than  this  one.  ” 

• “It  is  useful  to  focus  on  SLO  to  ensure  course  content  and  exam  matches  with  [each  other?].  ” 

3.  Did  the  presentation  and  handouts  provide  appropriate  information  to  guide  you  in  the  development 
of  student  learning  outcomes  without  overwhelming  you  with  too  much  material? 

Yes  6/7  No,  not  really  1/7 

Why? 

• “Presentation  was  confusing-so  much  terminology  for  more  or  less  the  same  thing.  Paper- 
splitting hairs  it  seems.  ” 

• “Not  much  of  a presentation  today.  But  thought  was  useful  from  last  time.  ” 

4.  Did  the  “student  learning  outcome”  sessions  help  you  to  specify  the  knowledge,  skills,  and  ability 
course  content  more  clearly  and  to  identify  priorities  more  logically? 

Yes  7/7  0/7  No,  not  really 

Why? 

• “I  had  already  done  a pretty  good  job  of  content  identification  and  prioritization.  ” 

• “However,  I would  complement  it  with  looking  at  other  complete  examples.  ” 

• “Ensure  appropriate  issues  related  to  knowledge,  issues,  etc.  are  addressed.  ” 

• “Although  I had  much  of  it  [in]  my  head-I  had  not  translated  it  to  ‘paper’  or  organized  it  fully.  ” 
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5.  Did  the  “student  learning  outcome”  session  help  you  to  develop  new  or  enhanced  student  learning 
objectives? 

6/7  Yes,  they  are  more. . . (select  all  that  apply)  1/7  Not  really 

1/7  a.  intentional  in  content  and  result  or  outcome 

1/7  b.  results  oriented  - outcomes  oriented  in  that  they  clearly  state  what  students  are  to  know  about, 
know,  or  be  able  to  do 

1/7  c.  specific  in  what  knowledge,  skill,  ability  is  to  be  learned  or  extended  by  the  student 
1/7  d.  measurable 

e.  observable 

1/7  f.  appropriately  stated  - using  more  definite  verbs  and  nouns;  they  explain  the  purpose,  provide 
context,  situation,  conditions,  etc. 

5/7  g.  all  of  the  above 

1/7  no  selections  (responded  Yes  above) 

Why? 

• “Not  so  much  for  the  specific  course,  more  for  another  course  I’m  teaching.  Exercise  brought 
organizational  clarity.  ” 

6.  In  completing  the  calendar  for  the  formal  scheduling  of  course  content,  there  is  greater  potential  to 

(select  all  that  apply) 

1/7  a.  enhance  or  improve  the  course  focus 

1/7  b.  provide  a better  format  for  on-going  critical  analysis  of  the  course  content  as  updates  or  changes  are 

needed 

1/7  c.  enhance  or  improve  the  course  content  delivery 

1/7  d.  help  me  better  visualize  my  course  and  how  to  continuously  update,  improve  or  enhance  it  to 

continuously  increase  student  learning 

1/7  e.  provide  the  students  with  a clearer  picture  of  the  course  and  what  they  are  to  learn 

1/7  f.  help  me  and  my  students  to  stay  on  “course” 

6/7  g.  all  of  the  above 

Why? 

7.  Now  that  you  have  written  student  learning  outcomes,  do  you  feel  more  able  to  prepare  learning 
activities  or  experiences? 

5/7  Yes  1/7  [wrote  in  Somewhat]  0/7  No,  not  really 

Why? 

• “Not  so  much  for  my  course.  More  for  another  course.  ” 

8.  Would  you  recommend  that  other  faculty  members  get  the  opportunity  to  engage  in  revising  their 
course  Student  Learning  Outcomes  - content  identification  and  student  learning  outcomes? 

5/7  Yes  2/7  No  Response  0/7  Not  really 

Why? 

• “Mentioned  above  ‘Not  so  much  for  this  course.  More  for  another  course.  ’ ” 

• “Too  soon  to  use  until  we  finish  our  program  and  further  explore  its  meaning” 
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9.  Was  the  “process”  used  during  these  days  effective? 

6/7  Yes  1/7  No  Response  0-/7  Not  really 

Why? 

10.  Strengths  of  this  aspect  of  the  program. 

• “It’s  worth  the  time  so  far.  ” 

• “Really  aided  in  focusing  our  efforts  to  see  an  improved  end.  ” 

• “Was  able  to  identify  where  the  breakdown  is  in  communication  between  faculty  and  students. 

11.  Areas  to  improve  in  this  aspect  of  the  program. 

12.  General  Comments: 

• “Great!” 

• “ Excellent  day” 
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NOTE:  Only  6/7  respondents  as  one  member  had  to  leave  as  feedback  forms  were  handed  out. 

CEET  Initiative  on  Teaching  and  Learning 
Gap  Analysis  Summary  and  Test  Analysis  Feedback-  Day  3 - Feb.  16,  2006 

1.  Was  today  worth  your  time  - worthwhile? 

Yes  6/6  Not  really 

Why? 

• "I  really  liked  the  analysis  of  tests” 

• "Learned  how  to  analyze  the  test,  like  discrimination  indexes  " 

• "Allow  to  understand  test  profile" 

• "Test  stats  interesting" 

2.  Would  you  recommend  today’s  content  for  other  faculty  members? 

Yes  5/6  1/6  Not  really 

Why? 

3.  Did  the  GAPs  Analysis  & Summary  help  you  to  see  possibilities  for  extending  or  enhancing:  course 
content  and  teaching/learning  strategies.  Also  did  it  make  you  aware  of  models  and  techniques  that  you 
could  consider  using  to  build  or  extend  student  learning  experiences  to  higher  levels  of  learning? 

Yes  6/6  Not  really 

Why? 

• "More  insight  into  course  delivery" 

• "I  essentially  did  this  last  week.  I guess  the  new  form  that  organized  info  differently  was  good" 
[relates  to  GAP  summary] 

• "This  will  help  to  incorporate  additional  activities  within  the  class" 

4.  Would  you  recommend  that  other  faculty  members  get  the  opportunity  to  engage  in  the  preliminary 
aspect  of  test  analysis? 

Yes  5/6  1/6  Not  really  - too  soon  to  use  until  we  finish  our  program  and  further  explore  its 

meaning 

Why? 

• "Could  be  valuable  " 

5.  Was  the  “process”  used  today  effective? 

Yes  6/6  Not  really 

Why? 

• "Contains  both  activity  and  lecture" 

6.  Strengths  of  today’s  program. 

• "analysis  of  tests" 

• "hands  on  " 

• "excellent  ways  to  look  @ test  from  a deeper  perspective  " 

• "working  with  real  data  " 
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7.  Areas  to  improve  today’s  program. 

• "Most  of  concepts  are  useful.  However,  learning  styles  probably  have  best  usefulness  because  no 
matter  what  is  the  type  of  student,  the  concept  needs  to  be  taught" 

• "Most  test  analysis  we  discussed  seemed  to  be  geared  toward  tests  with  many  questions... But  my 
test(s)  have  few  questions" 

• "none " 

8.  General  Comments: 

• "I  feel  today  I learned  something  very  important  to  analyze  my  teaching  work. -It  is  always 
difficult  to  analyze  tests,  but  today  was  a super  important  class  for  me  in  that  sense" 

• "none" 
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NOTE:  Only  6/7  participants  provided  feedback. 

CEET  Initiative  on  Teaching  and  Learning 
Analysis  Feedback-  Day  2 

1.  Was  today  worth  your  time  - worthwhile? 

Yes-6/6  Not  really 

Why? 

• “It  forced  us  to  take  a hard  look  at  the  current  state  of  our  courses” 

• “got  an  understanding  of  what  I was  doing  ” 

• “I  liked  dissecting  the  course  ” 

• “dissed  the  conpe”  [I  think  is]  “discussed  the  course” 

2.  Would  you  recommend  today’s  content  for  other  faculty  members? 

Yes -5/6  Not  really  1/6  (“May  Be”) 

Why? 

• “valuable” 

• “dissect  their  course”  [I  think  is]  “discussed  the  course” 

3.  Would  you  recommend  that  other  faculty  members  get  the  opportunity  to  engage  in  this  preliminary 
analysis? 

Yes  - 5/6  “If  there  is  an  opportunity”  Not  really  — 1/6  {too  soon  to  use  until  we  finish  our  program) 

4.  Was  the  “process”  used  today  effective? 

Yes  -5/6  Not  really  1/6 

Why? 

• “systematic 

• “you  may  use  the  O.H.  projector  to  explain  what  should  be  done”  approach” 

• “though,  still  I have  some  things  not  too  clean”  ]finish probably] 

• “worked  at  own  pace  ” 

5.  Strengths  of  today’s  program. 

• “Looking  at  various  aspects  of  learning  and  teaching” 

• “Well  thought” 

• “The  approach  ]I  think]  to  work  on  our  own  task” 

• “Good  flow;  materials  provided  were  easy  to  understand;  excellent  input  from  Jule” 

• “Hands  on  ” 

6.  Areas  to  improve  today’s  program. 

• “There  could  be  an  example  case  to  illustrate  the  process” 

• “I  would  like  to  know  how  to  apply  the  ‘Reversed  Instructional  Design  Method”  ]I  think] 

• “I  wish  I would  have  known  about  forms  ~^I  would  have  brought  my  laptop  ” 

• “I’m  one  of  those  learners  who  likes  to  see  big  picture  before.  It’s  slowly  coming  into  focus” 

• “Do  as  last  time.  Use  O.H.  ” 

1.  General  Comments: 

• “Very productive  day” 
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CEET  Initiative  on  Teaching  and  Learning 
Orientation  Feedback  Summary  (8/8  present) 

Feb.  2,  2006 

Feedback  inquiries  will  change  across  the  different  types  of  modules  with  some  constant 
items.  However,  both  content  assessment  and  quality  feedback  instruments  will  be  used 
throughout  the  program.  Below  is  a summary  of  faculty  feedback  offered  by 
participants  for  the  one-day  orientation  session. 

1.  Was  today  worth  your  time  - worthwhile? 

Yes  8/8  Not  really 

Why? 

• "Got  me  interested  in  learning  more  about  assessment" 

• "New  ideas  brought  to  light" 

• "It  helped  realize  where  I should  improve  " 

• "Force  to  think  how  to  improve  teaching" 

• "Laid  a background  for  the  program" 

2.  Would  you  recommend  today's  content  for  other  faculty  members? 

Yes  6/8  Not  really  1/1  and  1/1  "not  at  this  point" 

Why? 

• "Can  really  broaden  perspectives  " 

• "Not  a whole  lot  of  substance  today" 

• "It  helps  in  developing  good  teaching  capabilities  " 

• "Help  them  also  on  how  to  improve  teaching" 

3.  Will  the  survey  data  be  a useful  guide  for  instructional  decision-making? 

Yes  8/8  Not  really 

Why? 

• "Can  get  a baseline" 

• "I  think  it  might" 

• "to  identify  strengths  and  weaknesses" 

• "Even  though  it  should  be  corrected  by  the  number  of  students  answered  each 
question  " 

• "will  help  me  to  plan  my  course" 
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4.  Would  you  recommend  that  other  faculty  members  get  the  opportunity  to  use  the 
student  survey? 

Yes  5/8  and  1/8  Not  really  1/8 

Why? 

• "Yes,  but  maybe  wait" 

• “Too  soon  to  use  until  we  finish  our  program  and  further  explore  its  meaning” 

• "Fm  not  sure  it's  valuable  yet” 

5.  Was  the  "process"  used  today  effective? 

Yes  7/8  Not  really  1/8 

Why? 

• "Learn[ed]  detailed  analysis  of  teaching" 

• "for  an  introduction,  group  discussion  was  very  informative  " 

6.  Strengths  of  today's  program. 

• "The  data  generated  from  the  survey  has  helped  us  understand  our  teaching  and 
improve  on  them  " 

• "Good  organization  " 

• "Everything " 

• "New  teaching  methods  discussed" 

• "Instructors.  I think  they  are  prepared  to  conduct  this  initiative  and  I look 
forward  to  learning  how  to  improve  my  teaching  skills  " 

• "Individual  and  group  discussion  " 

• "Got  me  interested  in  learning  more  about  assessment" 

• "Good  overall  overview  & objectives  presented" 

7.  Areas  to  improve  today's  program. 

• "Allowing  more  time  for  reviewing  the  survey  result" 

• "Can't  say  yet" 

• "Good  as  is" 

• "Reduce  lunch  time  & get  out  early  " 

• "None" 

• "First  half  of  day:  Lot's  of  definitions  that  don't  mean  a whole  lot" 

8.  General  comments: 

• "I'm  excited,  I see  so  much  potential" 

• "None" 

• "Good  group  to  work  with" 

• "Excellent  workshop" 

• "I  think  this  is  something  very  important  for  faculty  members,  specially  those  that 
are  just  starting" 
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RESULTS:  END-OF-COURSE  QUESTIONNAIRE  AND  STANDARD 

EVALUATIONS 

(See  Program  A. 5-  and  Instruments  in  C - Scarborough) 

Jule  Dee  Scarborough,  Ph.D.  and  Jerry  Gilmer,  Ph.D. 

A questionnaire  was  developed  to  assess  the  students’  and  professors’  perceptions  of  the 
overall  quality  of  instruction,  course  content,  and  the  learning  environment  across  the  college 
(Scarborough,  2006).  The  questionnaire  included  questions  about  different  areas  of 
instruction  such  as  course  objectives  and  testing  activities.  The  instrument  was  completed 
along  with  the  regular  end-of-course  evaluations  of  instruction  in  the  professors’  classes  at 
the  end  of  the  Fall  2005  semester  and  again  at  the  end  of  the  Fall  2006  semester,  and  the  data 
between  the  two  years  were  compared.  The  questionnaire  was  from  the  perspective  of  the 
student  and  primarily  intended  for  student  feedback.  However,  we  also  used  it  with  the 
professors  and  asked  them  to  complete  it  from  their  perception  of  the  students’  perspective 
about  their  course,  instruction,  assessments,  etc.  It  was  interesting  to  compare  the  students’ 
perceptions  to  the  professors’  perceptions  of  how  the  students  might  respond. 

Important  Note:  Although  many  of  the  significance  levels  are  less  than  .05  (a  traditional 
indicator  of  statistical  significance),  due  to  the  length  and  complexity  of  the  questionnaire 
and  the  nature  of  the  students’  actual  responses,  the  validity  of  the  data  might  be 
questionable.  The  person  scanning  the  answer  sheets  on  which  the  students  bubbled  their 
responses  provided  the  following  warning:  “I  had  a lot  of  trouble  scanning  these  sheets 
because  of  multiple  marks;  just  about  half  of  the  sheets  were  rejected  by  the  scanner  because 
of  multiple  marks  where  we  were  not  expecting  (or  accepting)  them.  I found  this  with  both 
the  student  and  faculty  groups.  My  impression  is  the  people  filling  out  this  evaluation  were 
not  paying  attention  to  what  they  were  doing  or  not  closely  reading  the  questions  and 
instructions.  I also  remember  having  the  same  problem  with  the  sheets  last  December.”  In  all 
cases  where  there  appears  to  be  statistical  significance,  the  2006  scores  were  higher  than  the 

2005  scores.  However,  if  the  students  are  not  paying  attention  to  the  actual  questions  and 
simply  filling  in  bubbles  on  the  answer  sheet,  the  scores  will  tend  to  be  higher  than  they 
would  otherwise  due  to  the  scoring  methodology. 

Also  Important  Note:  When  we  reported  what  the  person  scanning  answer  sheets  indicated 
(above)  to  the  professors  and  asked  them  about  the  questionnaire  administration,  they  felt 
that  it  was  administered  the  same  as  it  was  in  the  2005  classes,  that  the  time  allocated  was 
approximately  the  same,  that  the  students  did  take  it  seriously,  and  that  the  students  seemed 
to  focus  when  completing  it.  They  also  felt  that  they,  themselves,  took  it  seriously  and  that 
they  completed  the  questionnaire  as  they  should  have.  They  wondered  why  the  person 
scanning  did  not  mention  his  doubts  about  the  2005  questionnaire  then,  rather  than  now,  and 
questioned  memory  as  reliable.  The  professors  all  felt  that  the  results  for  both  the  2005  and 

2006  questionnaires  could  be  used  without  suspicion  and  that  the  results  provided  good 
information  and  feedback.  A summary  of  the  statistical  significance  levels  for  the  scores 
from  the  professors,  all  students  combined,  and  the  cohort  of  students  from  each  professor’s 
class  is  presented  in  the  table  below. 

See  Instrument  on  pages  9-24  below.  A copy  without  point  values  is  in  Volume  III,  Portfolio 
Section  C.IO 
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Table  B.4.1 : Statistical  Significance  Levels  Comparing  Scale  Means  Between  Scores  from  Fall  2005-  Fall 
2006 


Results  from  All 
Professors  and  All 
Students 

Results  from  Students  in  Each  Professor’s  Class 

Content  Area 

All  7 

Professors 

All 

Students 

Ibrahim 

Abdel- 

Motaleb 

Abul 

Azad 

Briano 

Coller 

Abhijit 

Gupta 

Reinaldo 

Moraga 

Regina 

Rahn 

Robert 

Tatara 

Objectives  & Syllabus 
Items  (101-104) 

0.005 

0.407 

0.975 

0.971 

0.708 

0.577 

0.114 

0.023 

0.581 

Testing  & Measurement 
Items  (105-111) 

0.000 

0.000 

0.000 

0.420 

0.002 

0.000 

0.009 

0.000 

0.286 

Learning  & Teaching  Methods 
Items  (112-117) 

0.010 

0.000 

0.001 

0.150 

0.210 

0.008 

0.134 

0.000 

0.222 

Cooperative  & Group  Learning 
Items  (118-124) 

0.020 

0.000 

0.000 

0.451 

0.071 

0.487 

0.008 

0.019 

0.000 

Language 

Item  (125) 

0.361 

0.896 

0.690 

0.028 

0.441 

0.805 

0.308 

0.489 

0.511 

All  Engineering  Courses 
Items  (126-136) 

0.754 

0.978 

0.108 

0.586 

0.133 

0.574 

0.392 

0.416 

Project  Total 

Items  (101-125) 

0.000 

0.000 

0.000 

0.402 

0.024 

0.038 

0.109 

0.000 

0.075 

Overall  Total 

Items  (101-136) 

0.000 

0.000 

0.418 

0.044 

0.019 

0.099 

0.001 

0.050 

Shaded  cells  indicate  statistical  significance  beyond  the  .05  level. 
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The  following  tables  present  the  scale  means  and  standard  deviations  for  each  scale  separately 
for  the  professors’  responses  and  for  the  students’  responses. 


Table  B.4.2:  Professors 


Scale 

Year 

N 

Mean 

SD 

Sig.  Level 

Objectives  & Syllabus  (101-104) 

Fall  2006 
Fall  2005 

7 

7 

18.9 

15.6 

1.3 

2.1 

0.005 

Testing  & Measurement  (105-111) 

Fall  2006 
Fall  2005 

7 

7 

21.4 

13.4 

2.2 

2.3 

0.000 

Learning  & Teaching  Methods  (112-117) 

Fall  2006 
Fall  2005 

7 

7 

21.1 

13.4 

4.3 

5.1 

0.010 

Cooperative  & Group  Learning  (118-124) 

Fall  2006 
Fall  2005 

7 

5 

14.6 

6.6 

3.9 

6.2 

0.020 

Language  (125) 

Fall  2006 
Fall  2005 

7 

7 

3.6 

3.1 

0.5 

1.1 

0.361 

All  Engineering  Courses  (126-136) 

Fall  2006 
Fall  2005 

0 

0 

Project  Total  (101-125) 

Fall  2006 
Fall  2005 

7 

5 

79.6 

51.4 

9.3 

7.9 

0.000 

Overall  Total  (101-136) 

Fall  2006 
Fall  2005 

0 

0 

Table  B.4.3:  Students 


Scale 

Year 

N 

Mean 

SD 

Sig.  Level 

Objectives  & Syllabus  (101-104) 

Fall  2006 
Fall  2005 

146 

160 

12.8 

12.4 

5.0 

5.1 

0.407 

Testing  & Measurement  (105-111) 

Fall  2006 
Fall  2005 

134 

146 

17.8 

12.7 

3.7 

3.4 

0.000 

Learning  & Teaching  Methods  (112-117) 

Fall  2006 
Fall  2005 

129 

138 

16.8 

12.5 

7.6 

5.9 

0.000 

Cooperative  & Group  Learning  (118-124) 

Fall  2006 
Fall  2005 

131 

106 

10.3 

6.4 

4.3 

6.1 

0.000 

Language  (125) 

Fall  2006 
Fall  2005 

132 

145 

3.3 

3.3 

1.0 

1.0 

0.896 

All  Engineering  Courses  (126-136) 

Fall  2006 
Fall  2005 

120 

129 

14.9 

14.6 

5.6 

6.3 

0.754 

Project  Total  (101-125) 

Fall  2006 
Fall  2005 

122 

90 

62.3 

48.9 

14.7 

15.3 

0.000 

Overall  Total  (101-136) 

Fall  2006 
Fall  2005 

113 

82 

78.1 

62.4 

17.1 

17.0 

0.000 
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Table  B.4.4:  Professors  - SPSS  Output 


Group  Statistics 


Scale 

Year 

N 

Mean 

Std. 

Deviation 

Std.  Error 
Mean 

Objectives  & Syllabus  (101-104) 

Fall  2006 

7 

18.8571 

1.34519 

0.50843 

Fall  2005 

7 

15.5714 

2.14920 

0.81232 

Testing  & Measurement  (105-111) 

Fall  2006 

7 

21 .4286 

2.22539 

0.84112 

Fall  2005 

7 

13.4286 

2.29907 

0.86897 

Learning  & Teaching  Methods  (112-117) 

Fall  2006 

7 

21.1429 

4.29839 

1 .62464 

Fall  2005 

7 

13.4286 

5.09435 

1 .92548 

Cooperative  & Group  Learning  (118-124) 

Fall  2006 

7 

14.5714 

3.86683 

1.46152 

Fall  2005 

5 

6.6000 

6.18870 

2.76767 

Language  (125) 

Fall  2006 

7 

3.5714 

0.53452 

0.20203 

Fall  2005 

7 

3.1429 

1.06904 

0.40406 

All  Engineering  Courses  (126-136) 

Fall  2006 

0 

Fall  2005 

0 

Project  Total  (101-125) 

Fall  2006 

7 

79.5714 

9.25306 

3.49733 

Fall  2005 

5 

51.4000 

7.89303 

3.52987 

Overall  Total  (101-136) 

Fall  2006 

0 

Fall  2005 

0 
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Table  B.4.5:  Professors  - SPSS  Output 


Independent  Samples  Test 


Levene's  Test 
for  Equality  of 
Variances 

t-test  for  Equality  of  Means 

F 

Sig. 

t 

df 

Sig. 

(2- 

tailed) 

Mean 

Difference 

Std.  Error 
Difference 

95%  Confidence 
Interval  of  the 
Difference 

Lower 

Upper 

Objectives  & 
Syllabus  (101- 
104) 

Equal  variances 
assumed 

0.658 

0.433 

3.429 

12 

0.005 

3.28571 

0.95831 

1.19773 

5.37370 

Equal  variances 
not  assumed 

3.429 

10.076 

0.006 

3.28571 

0.95831 

1.15262 

5.41880 

Testing  & 

Measurement 

(105-111) 

Equal  variances 
assumed 

0.048 

0.830 

6.615 

12 

0.000 

8.00000 

1 .20937 

5.36500 

10.63500 

Equal  variances 
not  assumed 

6.615 

11.987 

0.000 

8.00000 

1 .20937 

5.36469 

10.63531 

Learning  & 
Teaching 
Methods  (112- 
117) 

Equal  variances 
assumed 

0.005 

0.943 

3.062 

12 

0.010 

7.71429 

2.51931 

2.22517 

13.20340 

Equal  variances 
not  assumed 

3.062 

11.670 

0.010 

7.71429 

2.51931 

2.20789 

13.22069 

Cooperative  & 
Group  Learning 
(118-124) 

Equal  variances 
assumed 

3.689 

0.084 

2.762 

10 

0.020 

7.97143 

2.88591 

1.54122 

14.40164 

Equal  variances 
not  assumed 

2.547 

6.219 

0.042 

7.97143 

3.12986 

0.37796 

15.56489 

Language  (125) 

Equal  variances 
assumed 

0.809 

0.386 

0.949 

12 

0.361 

0.42857 

0.45175 

-0.55572 

1.41286 

Equal  variances 
not  assumed 

0.949 

8.824 

0.368 

0.42857 

0.45175 

-0.59649 

1 .45363 

All  Engineering 
Courses  (126- 
136) 

Equal  variances 
assumed 

0.048 

0.830 

5.508 

10 

0.000 

28.17143 

5.11440 

16.77584 

39.56702 

Equal  variances 
not  assumed 

5.669 

9.564 

0.000 

28.17143 

4.96903 

17.03089 

39.31197 

Project  T otal 
(101-125) 

Equal  variances 
assumed 

0.658 

0.433 

3.429 

12 

0.005 

3.28571 

0.95831 

1.19773 

5.37370 

Equal  variances 
not  assumed 

3.429 

10.076 

0.006 

3.28571 

0.95831 

1.15262 

5.41880 

Overall  Total 
(101-136) 

Equal  variances 
assumed 

0.048 

0.830 

6.615 

12 

0.000 

8.00000 

1 .20937 

5.36500 

10.63500 

Equal  variances 
not  assumed 

6.615 

11.987 

0.000 

8.00000 

1 .20937 

5.36469 

10.63531 
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Table  B.4.6:  Students  - SPSS  Output 


Group  Statistics 


Scale 

Year 

N 

Mean 

Std. 

Deviation 

Std.  Error 
Mean 

Objectives  & Syllabus  (101-104) 

Fall  2006 

146 

12.8493 

4.98874 

0.41287 

Fall  2005 

160 

12.3688 

5.12439 

0.40512 

Testing  & Measurement  (105-111) 

Fall  2006 

134 

17.8060 

3.66548 

0.31665 

Fall  2005 

146 

12.6507 

3.44332 

0.28497 

Learning  & Teaching  Methods  (112-117) 

Fall  2006 

129 

16.7597 

7.60425 

0.66952 

Fall  2005 

138 

12.4638 

5.94151 

0.50578 

Cooperative  & Group  Learning  (118- 
124) 

Fall  2006 

131 

10.2672 

4.32137 

0.37756 

Fall  2005 

106 

6.4151 

6.12080 

0.59450 

Language  (125) 

Fall  2006 

132 

3.2879 

0.96922 

0.08436 

Fall  2005 

145 

3.3034 

1 .00225 

0.08323 

All  Engineering  Courses  (126-136) 

Fall  2006 

120 

14.8500 

5.57553 

0.50897 

Fall  2005 

129 

14.6124 

6.31752 

0.55623 

Project  Total  (101-125) 

Fall  2006 

122 

62.3443 

14.74341 

1.33481 

Fall  2005 

90 

48.9333 

15.25793 

1 .60833 

Overall  Total  (101-136) 

Fall  2006 

113 

78.1062 

17.13151 

1.61160 

Fall  2005 

82 

62.4268 

16.96185 

1.87312 
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Table  B.4.7:  Students  - SPSS  Output 


Independent  Samples  Test 


Levene's  Test 
for  Equality  of 
Variances 

t-test  for  Equality  of  Means 

F 

Sig. 

t 

df 

Sig. 

(2- 

tailed) 

Mean 

Difference 

Std.  Error 
Difference 

95%  Confidence 
Interval  of  the 
Difference 

Lower 

Upper 

Objectives  & 
Syllabus  (101- 
104) 

Equal  variances 
assumed 

0.139 

0.709 

0.830 

304 

0.407 

0.48057 

0.57914 

-0.65907 

1 .62020 

Equal  variances 
not  assumed 

0.831 

302.718 

0.407 

0.48057 

0.57843 

-0.65769 

1.61882 

Testing  & 

Measurement 

(105-111) 

Equal  variances 
assumed 

2.549 

0.111 

12.134 

278 

0.000 

5.15529 

0.42486 

4.31894 

5.99163 

Equal  variances 
not  assumed 

12.102 

272.016 

0.000 

5.15529 

0.42600 

4.31661 

5.99396 

Learning  & 
Teaching 
Methods  (112- 
117) 

Equal  variances 
assumed 

8.252 

0.004 

5.162 

265 

0.000 

4.29592 

0.83225 

2.65727 

5.93458 

Equal  variances 
not  assumed 

5.120 

242.111 

0.000 

4.29592 

0.83908 

2.64309 

5.94876 

Cooperative  & 
Group  Learning 
(118-124) 

Equal  variances 
assumed 

29.849 

0.000 

5.667 

235 

0.000 

3.85208 

0.67972 

2.51297 

5.19120 

Equal  variances 
not  assumed 

5.470 

182.766 

0.000 

3.85208 

0.70426 

2.46255 

5.24161 

Language  (125) 

Equal  variances 
assumed 

0.041 

0.840 

-0.131 

275 

0.896 

-0.01557 

0.11870 

-0.24924 

0.21810 

Equal  variances 
not  assumed 

-0.131 

273.987 

0.896 

-0.01557 

0.11851 

-0.24887 

0.21773 

All  Engineering 
Courses  (126- 
136) 

Equal  variances 
assumed 

2.354 

0.126 

0.314 

247 

0.754 

0.23760 

0.75736 

-1.25411 

1 .72930 

Equal  variances 
not  assumed 

0.315 

246.330 

0.753 

0.23760 

0.75395 

-1.24742 

1.72261 

Project  T otal 
(101-125) 

Equal  variances 
assumed 

0.030 

0.862 

6.450 

210 

0.000 

13.41093 

2.07924 

9.31208 

17.50978 

Equal  variances 
not  assumed 

6.416 

188.166 

0.000 

13.41093 

2.09008 

9.28794 

17.53392 

Overall  Total 
(101-136) 

Equal  variances 
assumed 

0.087 

0.768 

6.335 

193 

0.000 

15.67937 

2.47493 

10.79798 

20.56075 

Equal  variances 
not  assumed 

6.345 

175.683 

0.000 

15.67937 

2.47100 

10.80270 

20.55603 

See  Instrument  on  pages  9-24 
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TRADITIONAL  END-OF-COURSE  EVALUATIONS 
Jule  Dee  Scarborough,  Ph.D.  and  Jerry  Gilmer,  Ph.D 

The  traditional  end-of-course  evaluations  were  only  indirectly  related  to  the  components  and 
content  of  this  project.  For  that  reason  and  due  to  the  number  and  variety  of  factors  that  can 
affect  end-of-course  evaluations,  confounding  year-to-year  comparisons,  no  analyses  involving 
these  evaluations  was  performed. 

It  is  suggested  that  in  the  round  of  classroom  research  that  professors  design  the  research  to 
include  the  traditional  course  evaluation  a 
s a factor  in  the  research. 
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Traditional  end  of  course  evaluations  - Fall  2005 


Item 


# of  Sheets 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

Average 

Ibrahim 

Abdel-Motaleb 

14 

4.50 

2.43 

3.50 

4.14 

3.36 

3.64 

4.43 

3.57 

3.36 

3.93 

3.38 

3.66 

Abul 

Azad 

11 

4.73 

4.36 

4.27 

4.36 

4.27 

3.82 

4.36 

4.64 

4.64 

4.27 

4.55 

4.39 

Brianno 

Cellar 

54 

4.83 

4.23 

4.25 

4.53 

4.02 

3.52 

4.57 

4.72 

4.62 

4.17 

4.54 

4.36 

Abhijit 

Gupta 

37 

4.72 

3.53 

3.19 

3.58 

3.81 

3.06 

3.69 

4.06 

4.11 

3.94 

3.74 

3.77 

Reinaldo 

Moraga 

16 

3.88 

2.44 

2.44 

2.31 

1.69 

1.94 

3.00 

3.00 

3.06 

2.69 

2.00 

2.59 

Regina 

Rahn 

16 

4.88 

4.44 

4.31 

4.31 

4.50 

4.21 

4.75 

5.00 

4.94 

4.38 

4.56 

4.57 

Bob 

Tatara 

8 

4.88 

4.13 

4.25 

4.38 

4.13 

3.50 

3.88 

4.75 

4.50 

4.38 

4.38 

4.29 

Traditional  end  of  course  evaluations  - Fall  2006 


Item 


# of  Sheets 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

Average 

Ibrahim 

Abdel-Motaleb 

19 

4.63 

3.74 

3.53 

3.58 

3.63 

4.00 

4.32 

4.47 

4.16 

4.37 

3.68 

4.01 

Abul 

Azad 

12 

4.91 

3.55 

3.91 

4.09 

4.36 

3.64 

4.55 

4.00 

4.00 

4.36 

4.18 

4.14 

Brianno 

Cellar 

59 

4.90 

4.36 

4.16 

4.30 

3.86 

3.36 

4.45 

4.57 

4.53 

4.36 

4.48 

4.30 

Abhijit 

Gupta 

32 

4.72 

3.16 

3.06 

2.69 

2.69 

2.50 

3.59 

4.41 

4.47 

4.16 

3.52 

3.54 

Reinaldo 

Moraga 

14 

4.21 

3.29 

3.36 

3.79 

2.64 

3.29 

4.14 

4.21 

3.92 

4.00 

3.38 

3.66 

Regina 

Rahn 

14 

5.00 

4.71 

4.86 

4.71 

4.71 

3.86 

4.50 

5.00 

5.00 

4.43 

4.86 

4.69 

Bob 

Tatara 

19 

4.68 

3.89 

4.21 

3.84 

3.47 

3.63 

3.58 

4.37 

4.47 

4.32 

3.95 

4.04 

Fall  2005 
# of  Sheets 

Mean 

Fall  2006 
# of  Sheets 

Mean 

Ibrahim 

Abdel- 

Motaleb 

14 

3.66 

19 

4.01 

Abul 

Azad 

11 

4.39 

12 

4.14 

Brianno 

Collar 

54 

4.36 

59 

4.30 

Abhijit 

Gupta 

37 

3.77 

32 

3.54 

Reinaldo 

Meraga 

16 

2.59 

14 

3.66 

Regina 

Rahn 

16 

4.57 

14 

4.69 

Bob 

Tatara 

8 

4.29 

19 

4.04 
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PROFESSORS  — READ  OUT  LOUD  CAREFULLY  to  students  before  handing  out 
questionnaires. 


To:  Participating  students 

Fr:  Dean 

Re:  New  initiative  to  study  the  quality  of  instruction,  course  content,  and  the  learning 

environment  across  the  college 


The  following  questionnaire  is  being  administered  to  selected  classes  across  the  college  and  its 
four  departments.  As  students  in  those  classes,  you  are  being  asked  to  participate  in  providing 
baseline  information  about  the  quality  of  instruction,  course  content,  and  the  learning 
environment  in  the  college.  It  is  important  to  take  note  of  the  following: 

(a)  The  first  part  of  the  questionnaire  relates  only  to  this  course. 

(b)  The  second  part  of  the  questionnaire  relates  to  your  experience  across  all  the  courses  you 
have  taken  in  your  major  department. 

(c)  This  questionnaire  does  not  seek  information  about  your  experience  in  any  courses  outside 
the  major  department  (e.g.  general  education  or  courses  transferred  to  NIU). 

Your  responses  to  these  questions  will  be  used  as  baseline  information  to  study  how  to 
strengthen  the  quality  of  education  across  the  college. 

We  are  hoping  that  you  will  complete  this  questionnaire  thoughtfully,  seriously,  and  genuinely, 
with  the  understanding  that  it  is  important  and  will  assist  us  in  structuring  a college  initiative  to 
study  and  strengthen  the  quality  of  instruction,  course  content,  and  the  learning  environment 
across  the  college. 

In  testing  the  questionnaire  with  students,  it  took  about  20  minutes;  therefore,  we  are  allowing  30 
minutes  of  class  time  to  complete  the  questionnaire  in  class. 

The  questionnaire  is  somewhat  long,  but  not  as  long  as  it  may  seem  because  the  questions  have 
been  written  in  a way  that  hopefully  describe  thoroughly  what  we  are  seeking  information  about. 
Also  the  print  is  regular  sized,  and  we  have  spaced  and  printed  the  document  for  easier  reading. 

Please  attend  to  each  item  carefully  and  respond  to  the  best  of  your  ability.  We  need  your  input. 
It  is  important  that  you  respond  honestly,  genuinely,  and  with  sincerity  as  the  results  of  the 
survey  will  greatly  impact  the  Dean's  new  initiative  on  the  quality  of  education  for  students  in 
the  college. 

Thank  you  for  investing  your  time  and  serious  effort  to  help  us  begin  this  very  important 
initiative. 
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End  of  Course  Questionnaire  on  Teaching  and  Learning 

Jule  Dee  Scarborough  (2006) 


After  completing  the  student  and  course  information  on  the  front  side  of  the  scan  form,  respond  to  the  following 
questions  on  the  back  side  of  the  form  beginning  with  item  101. 

Questions  101-124  focus  on  the  course  you  are  now  ending.  Please  respond  to  101-124  based  upon 
your  experience  in  this  course  only. 

101.  The  course  syllabus  identified  specific  learning  objectives. 

2 a.  Yes,  and  I understood  them 
1 b.  Yes,  but  I didn't  understand  them 
0 c.  I don't  know 

0 d.  No,  there  were  no  learning  objectives 
Max  Points  Possible  — 2. 

102.  The  learning  objectives  for  this  course  were  chosen  or  required  by:  (Select  all  that  apply.) 

1 a.  Future  employers 
0 b.  Department  head 

0 c.  Professor's  interests 

1 d.  Accreditation  agency 

1 e.  NIU  General  Education  Goals 
Of.  I don't  know 
Max  Points  Possible  — 3 

103.  The  course  syllabus  specified:  (Select  all  that  apply) 

1 a.  course  or  student  learning  objectives 
1 b.  course  description 
1 c.  clearly  defined  course  content 

1 d.  clearly  defined  assignments,  labs,  papers,  projects,  tests,  or  other  important  assignments  or 
activities 

1 e.  the  course  schedule  or  timeline  identifying  meeting  dates,  assignment  due  dates,  and  the  semester's 
schedule 

1 f.  additional  explanations  of  course  requirements  that  established  the  criteria  for  each  assignment 
1 g.  references  other  than  the  text,  e.g.  books,  websites,  articles,  other  sources  related  to  course  content 
1 h.  contact  information  for  professor,  instructor,  and/or  graduate  teaching  or  lab  assistants 
Max  Points  Possible  — 8 
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104.  The  professor  (and  any  assistants):  (Select  all  that  apply) 

1 a.  focused  content  and  learning  activities  on  the  course  or  student  learning  objectives  throughout  the 
semester 

1 b.  provided  learning  that  seemed  to  align  with  the  course  description 

1 c.  taught  the  course  content  specified  in  the  syllabus 

1 d.  followed  the  assignments,  labs,  papers,  projects,  tests,  or  other  important  assignments  or  activities 
outlined  and  defined  in  the  syllabus 

1 e.  followed  the  course  schedule  or  timeline  specified  in  the  syllabus  (e.g.,  meeting  dates,  assignment 
due  dates,  and  the  semester's  schedule) 

1 f.  graded  assignments  according  to  the  written  explanations  for  course  requirements  establishing  the 
criteria  for  each  assignment 

1 g.  was(were)  available,  using  the  contact  information  for  professor,  instructor,  and/or  graduate 
assistants 

1 h.  deviated  from  the  syllabus  by  adding  appropriate  content  to  expand,  deepen  understanding,  or 
resolve  questions  resulting  in  adding  value  to  the  course;  any  additional  assignments  were 
appropriate  having  reasonable  timelines 

- 1 i.  deviated  from  the  syllabus  inappropriately  where  additions  to  the  information  provided  on  the 

syllabus,  or  new  assignments  added,  were  irrelevant  or  distracting  and  added  little  or  no  value  to 
the  course  or  learning;  new  assignments  were  untimely  and  caused  unnecessary  stress  for  students 

1 j.  The  course  was  well  organized,  structured,  and  executed. 

Max  Points  Possible  — 8 


105.  Which  of  the  following  methods  were  used  hy  the  professor  to  measure  learning?  (Select  all 
that  apply) 

1 a.  final  exam  - traditional  test 
1 b.  midterm  exam  - traditional  test 

1 c.  quizzes  and/or  short  tests  periodically-traditional  test(s)  (e.g.,  multiple-choice  or  true/false) 

1 d.  quizzes  and/or  short  tests  periodically  - short  answer  and/or  essay 

1 e.  research  or  learning  paper  (s),  usually  requiring  literature  search  or  field  research  and  formal  write- 
up 

1 f.  case  study(ies)  in  industry,  usually  requiring  a report  or  short  paper  write-up 
1 g.  hands-on  technical  project(s) 

1 h.  hands-on  non-technical  project(s) 

1 i.  other  types  of  performances,  "doing"  something 

1 j.  course  portfolio,  full  documentation  of  all  work  and  progress  in  the  course 
k.  other;  write  a description  here: 

Max  Points  Possible  — 10 
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106.  Select  ALL  the  descriptions  below  that  identify  the  methods  being  used  in  this  course  to 
measure  student  learning: 

1 a.  Learning  was  measured  on  my  ability  to  memorize  terminology,  symbols,  facts,  information, 

theory,  principles,  concepts,  information,  definitions,  descriptions 

2 b.  Learning  was  measured  on  my  ability  to  make  comparisons  to  determine  similar  and  dissimilar 

examples,  understanding  relationships  and  connections  between  and  among  facts,  concepts, 
theories,  principles,  translates  knowledge  into  a new  context,  interpret  facts,  predict 
consequences,  order,  group  information,  contrast,  distinguish,  estimate,  differentiate,  discuss,  or 
extend  knowledge 

3 c.  Learning  was  measured  on  my  ability  to  use  information,  methods,  concepts,  theories  in  new 

situations;  problem  solving  - this  requires  choosing  and  applying  knowledge  (e.g.,  the  best 
formula,  concept,  principle,  theory  to  solve  problems),  using  inductive  reasoning  to  determine 
the  best  methods,  techniques,  tools,  strategies  to  apply  towards  a best  solution;  this  method  of 
measurement  can  range  from  a test  item  with  a complex  problem  to  be  solved  or  a hands-on 
technical  problem  requiring  the  design  and  building  of  something  mechanical.  The  key  to  this 
method  is  that  it  requires  application  of  knowledge  - "doing"  (demonstrate,  calculate,  illustrate, 
show,  solve,  examine,  modify,  relate,  change,  experiment,  discover). 

4 d.  Learning  was  measured  on  my  ability  to  recognize  patterns  in  information,  problems,  and 

situations;  the  ability  to  organize  parts,  identify  or  discover  "hidden"  meanings,  and/or  identify 
components;  this  requires  one  to  analyze,  separate  thoughts,  processes,  problems,  order, 
explain,  connect,  classify,  and  divide,  compare,  select,  explain,  and/or  make  inferences  (indirect 
meanings);  this  requires  deductive  reasoning  where  one  begins  with  facts  and  information, 
makes  choices  to  gradually  discover  the  bigger  picture 

5 e.  Learning  measured  my  ability  to  hypothesize,  design,  support  argument,  schematize,  write, 

report,  justify,  choose,  evaluate,  estimate,  judge,  criticize,  defend,  use  old  ideas  to  create  new 
ones,  extending  the  old  idea  into  a new  one  for  extended  applications,  make  choices  based  upon 
reasoned  argument,  verify  value  of  evidence,  recognize  when  subjectivity  is  being  used  rather 
than  objectivity  (more  scientific),  make  sound  generalizations  from  given  facts,  relate  and  use 
knowledge  across  different  contexts,  predict  and  draw  conclusions,  combine,  integrate,  modify, 
rearrange,  substitute  knowledge,  plan,  formulate,  compare  and  discriminate  between, 
summarize,  and  make  conclusions 

6 f.  Learning  measured  my  ability  to  design,  discover,  invent,  develop,  create,  research;  transform 

knowledge  into  a product,  process,  technique,  model,  method,  strategy,  etc. 

Points  for  only  highest  level  response  only.  Max  Points  Possible  - 6 


13 


107.  Select  the  response  that  best  describes  the  relationship  between  the  traditional  tests  you  have 
taken  to  date  in  this  course  (e.g.  multiple-choice,  true/false  items,  etc.)  and  the  course  content . 

3 a.  the  content  of  the  test(s)  was  related  to  the  content  specified  in  the  syllahus,  and  only  to  content 
specified  in  the  syllabus. 

2 h.  the  content  of  the  test(s)  was  related  to  the  content  specified  in  the  syllabus  and  other  content 
proyided  by  the  professor  or  assistants. 

1 c.  the  content  of  the  test(s)  did  not  relate  to  the  content  specified  in  the  syllabus  but  did  relate  the 
other  content  proyided  by  the  professor  or  assistants. 

0 d.  the  content  of  the  test(s)  related  to  neither  (1)  the  content  specified  in  the  syllabus,  nor  (2)  the 
other  content  proyided  by  the  professor  or  assistants. 

Max  Points  Possible  — 3 

Items  108-111  relate  to  the  measurement  of  student  learning  through  performance(s)  rather  than 
traditional  tests.  ***  Consider  the  definitions  below  when  responding  to  items  108-111. 

***  Definitions: 

***Performance  Task  (or  assessment)  - any  authentic  or  real-world  task  designed  to  measure  student 
learning.  Such  a task  can  be  used  to  determine  what  students  can  "do"  with  knowledge.  Unlike  some 
traditional  tests,  performance  tasks  require  students  to  move  to  another  level  of  providing  evidence  of 
learning  - that  of  applying  or  using  knowledge  by  performing  authentic  tasks,  such  as  designing  a part  or 
product,  or  designing  and  then  producing  the  part  or  product.  Writing  a paper  would  provide  evidence  of 
research  skills  and  communication  skills,  for  example. 

(108)  Performance  tasks  were  used  to  measure  student  learning  in  this  course.  (*see  definition 
above) 

la.  Yes  (according  to  the  definition  above) 

0 b.  No  (according  to  the  definition  above) 

Max  Points  Possible  — 1 

***Rubric  - any  type  of  information  sheet  or  form,  check  off  sheet  that  establishes  the  levels  of 
performance  criteria  for  performance  tasks;  these  criteria  establish  standards  for  performance  and  the 
criteria  for  each  standard.  They  are  used  to  provide  students  information  about  what  is  required  to 
achieve  a particular  number  of  points  or  grade.  See  attached  example  at  end  of  questionnaire  following 
this  page;  then  continue  to  complete  the  questionnaire.. 

(109)  Rubrics  were  used  for  scoring  or  grading  the  performances  in  this  course. 

la.  Yes  (according  to  the  definition  below) 

0 b.  No  (according  to  the  definition  below) 

Max  Points  Possible  — 1 
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(110)  Below  are  examples  of  some  performance  tasks;  identify  any  that  are  similar  to 
performances  that  you  had  to  accomplish  during  this  course.  Select  all  that  apply: 

1 a.  Writing  a paper 

1 b.  Working  problems,  showing  the  entire  equation  worked  out  manually,  through  each  step  of  the 
equation 

1 c.  Designing  a product  part,  entire  machine,  other  major  design  project 
1 d.  Designing  an  industrial  production  system 
1 e.  Designing  electrical  circuitry  or  full  electrical/electronic  system 

1 f.  Designing  and  producing  a part  using  manufacturing  processes,  e.g.  actually  producing  product 
using 

manufacturing  production  equipment  in  a lab  or  on-site  in  industry 
Max  Points  Possible  — 6 

(111)  Select  all  examples  of  performance  tasks  helow  (similar)  where  a rubric  or  performance 
criteria  form  was  used  to  score  or  grade  the  performance(s)  during  this  course. 

1 a.  Writing  a paper 

1 b.  Working  problems,  showing  the  entire  equation  worked  out  manually,  through  each  step  of  the 
equation 

1 c.  Designing  a product  part,  entire  machine,  other  major  design  project 
1 d.  Designing  an  industrial  production  system 
1 e.  Designing  electrical  circuitry  or  full  electrical/electronic  system 

1 f.  Designing  and  producing  a part  using  manufacturing  processes,  e.g.  actually  producing  product 
using  manufacturing  production  equipment  in  a lab  or  on-site  in  industry 
Max  Points  Possible  — 6 
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112.  The  following  items  related  to  levels  of  learning  and  how  learning  takes  place. 

(Select  ALL  that  apply) 

1 a.  the  learning  of  basic  knowledge  requiring  me  to  list,  name,  identify,  show,  define, 

recognize,  recall,  state,  visualize,  state  facts,  concepts,  theories,  principles,  and/or 
information? 

2 b.  the  comprehension  or  greater  understanding  of  knowledge  through  activities  that  required 

me  to  summarize,  explain,  interpret,  describe,  compare,  paraphrase,  differentiate  , 
demonstrate,  classify,  or  contrast  facts,  information,  concepts,  theories,  principles? 

3 c.  the  application  or  opportunity  to  “do”  or  “perform,”  using  knowledge,  requiring  me  to 

solve  problems,  illustrate,  calculate,  use,  interpret,  relate,  manipulate,  apply,  modify 
facts,  concepts,  theories,  information,  or  data? 

4 d.  analytical  activities  that  required  me  to  analyze  and  organize  facts,  data,  and  information; 

deduce  patterns,  and  trends;  contrast,  compare,  distinguish,  differences  or  similarities; 
and  then  discuss  solutions,  directions  and  plan  or  devise  actions? 

5 e.  the  synthesis  and  evaluation  of  facts,  information,  data,  situations,  problems,  and 

furthermore  require  me  to  argue  rationally,  support  or  justify  a method,  solution,  action, 
choice  of  formula,  theory,  concept,  principle  or  result  in  the  need  to  propose  a hypothesis, 
following  with  the  design  of  an  experiment,  product,  process,  technique,  and/or  make 
judgments  that  had  to  be  critiqued  and  defended  and  finalized  into  reports,  summaries,  or 
papers. 

6 f.  the  design,  discovery,  invention,  development,  creation,  research,  or  transformation  of 

knowledge  into  products,  processes,  techniques,  models,  methods,  strategies,  etc.,  using 
design  and  development,  research,  experimentation,  and/or  development  knowledge, 
techniques,  procedures,  and  tools? 

Points  for  highest  level  only.  Max  Points  Possible  - 6 

113.  This  course  engaged  me  in  (Select  one  response) 

0 a.  learning  knowledge  and  skills  to  use  when  I get  a job. 

1 b.  learning  knowledge  and  skills  to  use  when  I get  a job,  but  also  provided  tbe  opportunity  to  apply 

that  knowledge  in  class  through  projects  or  activities  where  performing  tasks  using  that 
knowledge  were  required 
0 c.  neither  (a)  nor  (b),  very  well 
Max  Points  Possible  — 1 
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114.  The  following  list  identifies  and  briefly  describes  teaching  methods  the  professor  may 

use  during  instruction.  (Select  all  that  apply) 

1 a.  the  professor  lectures  information  and  connections;  I listen  and  take  notes,  if  I choose 

1 b.  the  professor  focuses  or  presents  content,  then  breaks  the  class  into  student  groups  to 

discuss  the  content,  then  engages  in  summarizing  and  clarifying  the  content  as  a group. 

1 c.  the  professor  focuses  or  presents  content,  then  assigns  individual  but  short  term  projects 
using  the  content  or  information,  e.g.  problem  to  solve,  design  project,  analysis. 

1 d.  the  professor  focuses  or  presents  content,  breaks  the  class  into  student  groups  to  discuss  the 
content,  and  then  engages  in  a short  term  group  project  using  the  content  or  information 
(e.g.,  problems  to  solve,  design  project,  analysis) 

1 e.  lessons  are  broken  down  in  components;  as  individual  students  master  each  component, 
they  are  tested.  When  they  pass  the  test,  they  go  on  to  the  next  component. 

1 f.  the  professor  uses  visual  charts,  displays,  a wide  range  of  graphic  organizers  or  other  visuals 
to  better  organize  and  present  information;  to  show  relationships  between  concepts  and 
principles;  and  to  increase  understanding  about  the  application  of  foundation  concepts  or 
principles. 

1 g.  when  presenting  content,  the  professor  uses  examples  that  are  and  are  not  representative  of 
the  concept  or  principle.  Students  compare  the  examples  and  match  those  that  represent 
the  concept  or  not;  gradually  as  more  examples  that  are  and  are  not  representative  are 
reviewed,  the  group  reaches  consensus  of  what  examples  directly  represent  the  content 
and  come  away  with  greater  understanding. 

1 h.  lessons  require  that  we  combine  concepts  and  analyze  the  relationships  of  concepts;  we 
then  engage  in  solving  problems. 

1 i.  during  the  lessons,  the  professor  asks  us  to  identify  and  enumerate  information  related  to 
concepts  as  they  are  demonstrated,  grouping  concepts  into  categories  with  common 
attributes. 

1 j.  we  learn  information  on  concepts  through  the  act  of  classification,  gathering  and 

classifying  information  to  build  and  test  hypotheses;  they  engage  in  experiments  and  the 
results  of  experiments  are  used  to  develop  hypothesis  generalizations  about  the  situation, 
idea,  or  problem. 

Max  Points  Possible  = 10 
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115.  The  following  list  identifies  and  briefly  describes  additional  teaching  methods  the 

professor  may  use  during  instruction.  (Select  all  that  apply) 

1 a.  students  are  presented  with  generalizations  and  examples  and  engage  in  trying  to  identify 
the  individual  situation  or  idea  that  is  embedded  (move  from  problem  to  why  something 
happens) 

1 b.  students  are  presented  with  a problem  and  then  create  questions  to  be  used  to  solve  the 
problem.  Students  engage  in  a process  of  investigation  and  explanation  of  the 
phenomena. 

1 c.  students  engage  in  a formally  organized  court  case  to  present  information  and  arguments 
about  the  ingrained  issues. 

1 d.  students  are  instructed  on  each  component  of  the  content,  and  all  must  be  successful  on  that 
content  before  the  professor  moves  on  with  new  or  more  complex  content 

1 e.  lessons  break  skills  down  into  components  and  sequences  of  action;  each  person  learns  the 
skill  step  by  step  the  same  way 

1 f.  lessons  begin  by  focusing  on  a current  situation;  analogies  are  used  to  define  the 

characteristics  of  the  situation;  analogies  continue,  using  other  graduated  analogies  until 
it  appears  to  have  no  relationship  to  the  origin;  the  lesson  then  uses  the  final  description 
of  the  analogy  to  compare  to  the  original  situation 

1 g.  lessons  engage  us  in  the  development  of  physical  skills,  such  as  welding 

1 h.  the  professor  uses  metaphors  to  make  content  more  familiar 

1 i.  lessons  focus  on  personal  development,  free  expression  of  ideas  and  feelings,  furthering 
your  self-understanding 

1 j.  students  explore  problems  through  actions  developing  problem  solving  skills;  we 
participate  and/or  observe 
Max  Points  Possible  = 10 
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116.  My  professor  exhibited  the  following  styles  of  instruction  throughout  the  semester.  (Select  all 

that  apply.) 

1 a.  professor  makes  all  decisions  on  what,  where,  when,  and  how  learning  takes  place; 

is  the  expert;  strives  for  precision,  synchronization,  and  uniformity;  determines 
what  is  taught  and  how  it  will  he  evaluated 

1 h.  students  are  given  a number  of  tasks  to  do  while  in  class;  students  can  ask  questions; 

professor  moves  around  and  gives  feedback 

1 c.  students  provide  feedback  to  each  other;  one  student  performs  while  another 

provides  feedback;  professor  designs  forms  to  guide  the  observations;  socialization 
is  inherent  in  this  style;  students  develop  feedback  skills 

1 d.  feedback  is  provided  by  you  as  the  individual  learner  to  yourself;  other  events  providing 
external  feedback  facilitate  your  ability  to  do  this;  professor  helps  you  become  a better 
evaluator,  thus,  increasing  your  self-esteem  about  working  independently 

1 e.  we  select  our  own  level  of  performance  and  alter  it  according  to  my/our  self-evaluation; 

the  professor  determined  the  tasks  and  defined  the  levels  of  difficulty 

1 f.  professor  leads  students  to  discover  concept  by  answering  a series  of  questions; 

professor  determines  concepts  and  best  sequences  for  guidance;  friendly  environment 
with  time  to  think  built  into  the  learning  opportunity;  professor  traces  a series  of 
questions  leading  to  the  answer 

1 g.  professor  presents  question;  students  use  logical  and  critical  thinking  to  discover 

solutions;  students  determine  questions  to  ask  rather  than  the  professor;  professor 
respects  the  student  process  and  dos  not  interfere 

1 h.  professor  encourages  students  to  find  multiple  solutions  to  given  problems;  professor 

selects  the  subject  and  designs  the  problem;  there  is  no  one  right  answer;  professor 
responds  to  student  process  rather  than  the  value  of  a solution  or  answer 

1 i.  the  student  and  professor  select  the  content  to  be  learned;  the  student  designs,  develops, 

and  performs  the  series  of  tasks  and/or  students  select  the  activity,  design  the 
experiences,  perform  the  tasks;  professors  assists/consults  with  the  evaluation  of  tasks 

1 j.  students  are  empowered  to  take  full  responsibility  for  the  learning  process;  they  are  not 

required  to  consult  with  the  professor 
Max  Points  Possible  — 10 
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117.  Which  of  the  following  best  descrihes  this  course? 

Choose  the  one  item  that  comes  closest  to  descrihing  your  experience  in  this  course. 

0  a.  The  professor  assumes  the  entire  responsibility  for  delivering  the  course  content.  He/she 
lectures  all  information  we  are  expected  to  learn.  The  text  is  used  as  a reference.  Lectures 
reflect  text  content. 

0 b.  The  professor  assumes  the  entire  responsibility  for  delivering  the  course  content  in 
combination  with  assigned  readings  from  the  textbook.  The  lectures  and  text  content 
provide  all  the  information  we  are  expected  to  learn.  Most  lectures  correlate  directly  or 
are  duplication  of  text  content. 

0 c.  Students  are  assigned  reading  from  the  text  to  gain  basic  course  content.  My  professor 

explains  difficult  content  from  the  text  and  then  adds  lectures  on  some  important  or 
critical  content  that  is  not  covered  in  the  text,  thus  expanding  or  deepening  understanding 
and  ability  to  use  the  information  from  the  text. 

1 d.  Students  are  responsible  for  some  of  their  own  learning.  For  example,  once  a concept  or 

principle  is  explained  by  the  professor  and  we  have  used  the  text  for  basic  learning,  as  a 
source  or  reference,  we  then  have  to  perform  research  on  content  ourselves  to  deepen 
our  understanding  of  the  concept  and  its  application  possibilities.  We  have  to  bring  the 
information  back  to  class  to  share  with  the  professor  and  class.  Student  activities  can  vary 
from  literature  research,  case  studies,  identifying  additional  sources  of  information,  e.g. 
books,  people,  examples,  demonstrations,  etc.  Students  are  required  to  learn  on  their  own 
or  in  small  groups  to  deepen  understanding  or  extend  learning  and  understanding  beyond 
that  presented  by  the  professor  or  established  learning  activities. 

2 e.  The  professor  assigns  reading  from  the  text,  explains  difficult  content,  and  then  provides 

content  to  deepen  or  extend  the  basic  text  content  or  to  clarify  or  explain  content  not  well 
understood.  Students  are  responsible  for  some  of  their  own  learning,  and  we  then  engage 
in  research  to  solidify  our  understanding  of  the  content.  Ultimately,  the  professor  then 
assigns  projects  that  expand  learning  into  the  “doing”  dimension  where  we  used  the 
content  learned  to  solve  a problem,  develop  a product,  construct  a theoretical  model,  use 
materials,  processes,  and  knowledge  to  create,  etc. 

3 f.  Students  are  responsible  for  a great  deal  of  their  own  learning.  After  working  with  us  in  a 

variety  of  ways,  many  of  them  are  highly  engaging  students  to  learn  important  knowledge 
and  skills  where  the  professor  is  more  of  a learning  coach,  direction  setter,  source  of 
validation,  someone  who  models  an  inquiry  driven  process  of  learning,  with  a strong 
focus  on  "how"  and  "why"  processes.  He/she  provides  the  opportunity  to  engage  in  the 
creation  of  a solution  to  an  identified  need  or  problem,  applying  the  knowledge  and  skills 
learned  earlier  or  throughout  the  learning  processes  throughout  the  semester. 

Max  Points  Possible  = 3 
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118.  This  course  provided  the  opportunity  to  work  cooperatively  in  small  groups  to 
accomplish  the  learning  of  course  content.  (Select  one) 

1 a.  Yes 

0 b.  No 

Max  Points  Possible  = 1 

119.  When  working  together,  we  sought  outcomes  that  benefited  me  individually  as  well  as 

the  whole  group.  (Select  one) 

2 a.  Most  of  the  time 

1 b.  Some  of  the  time 
0 c.  Not  really 

0 c.  No  opportunity  to  work  in  groups 
Max  Points  Possible  = 2 

120.  When  working  with  others,  I feel  that  we  maximized  my  own  learning  and  the 
learning  of  others.  (Select  one) 

2 a.  Most  of  the  time 

1 b.  Some  of  the  time 
0 c.  Not  really 

0 d.  No  opportunity  to  work  in  groups 
Max  Points  Possible  = 2 

121.  Working  in  groups  provided  greater  opportunity  for  everyone  to  learn  more  and 
resulted  in  higher  grades  for  all.  (Select  one) 

2 a.  Most  of  the  time 

1 b.  Some  of  the  time 
0 c.  Not  really 

0 d.  No  opportunity  to  work  in  groups 
Max  Points  Possible  = 2 

122.  When  you  were  required  to  work  in  student  groups  throughout  the  course,  were  those 
group  assignments  formally  organized  with  criteria  for  performance?  (Select  one) 

2 a.  Most  of  the  time 

1 b.  Some  of  the  time 
0 c.  Not  really 

0 d.  No  opportunity  to  work  in  groups 
Max  Points  Possible  = 2 

123.  When  you  were  required  to  work  in  student  groups  throughout  the  course,  did  the 
professor  provide  formal  and  specific  team  related  instruction  on  how  to  function 
effectively  and  productively  on  a team?  (Select  one) 

1 a.  Yes 
0 b.  No 

Max  Points  Possible  = 1 
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124.  Working  in  groups  results  in: 

(Select  as  many  as  apply  b-i;  if  you  choose  response  a,  move  on  to  question  125) 

0 a.  there  was  no  opportunity  to  work  in  groups 

(if  you  choose  this  selection,  move  on  to  question  125) 

1 b.  higher  achievement  and  productivity  by  all  or  almost  all  members  of  the  group 

1 c.  longer  term  retention  of  knowledge  being  learned 

1 d.  intrinsic  (inside  myself)  and  higher  motivation  to  achieve  by  all  or  almost  all 
members  of  the  group;  greater  focus  and  time  on  task 

1 e.  higher  level  thinking,  reasoning,  deeper  analysis  of  problems,  better  judgments 

1 f.  more  positive  relationships  between  most  students  or  among  group  members  and 
more  caring  about  each  other’s  learning  and  success;  feelings  of  more  support  in 
learning 

1 g.  greater  value  of  diversity  among  group  members;  greater  cohesion  among  students 
in  the  course 

1 h.  the  development  of  higher  self-esteem  among  most  students;  further  development 
of  self  identify 

1 i.  development  of  social  skills  so  that  students  learn  to  engage  with  each  other  in  a 
positive  manner,  even  when  conflicting  ideas  are  on  the  table 

1 j.  greater  ability  to  cope  with  adversity  and  stress 

Max  Points  Possible  = 9 

125.  The  professor's  language  skills  were  not  a barrier  in  communication  between 

the  professor  and  students. 

4 a.  Strongly  agree  - the  professor's  language  skills  were  exceptionally  good;  very 
effective  communication  took  place  between  the  professor  and  students. 

3 b.  Agree  - the  professor's  language  skills  were  good;  there  was  effective 
communication  between  the  professor  and  students. 

1 c.  Disagree  - the  professor's  language  skills  need  to  improve  for  effective 
communication  to  occur  between  the  professor  and  students. 

0 d.  Strongly  Disagree  - the  professor's  language  skills  were  inadequate  for  effective 

communication  between  the  professor  and  students;  poor  language  skills  resulted 
in  communication  barrier  between  the  professor  and  students. 

Max  Points  Possible  = 4 
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Unlike  Items  101-125  above  which  focused  on  THE  course  you  are  NOW  in  and 
completing,  the  following  questions  are  focused  more  broadly. 

For  Items  126-136,  reflect  on  your  experience  across  ALL  the  courses  you  have 
taken  in  engineering  and/or  technology  to  date.  Provide  your  perspective  by  generalizing 
across  ALL  the  courses  that  you  have  taken  in  engineering  and/or  technology  to  date  and 
respond  to  Items  125-135  below. 

126.  The  professors  teaching  the  engineering  and/or  technology  courses  that  I've  taken  to 
date  in  my  major:  (Select  one) 

3 a.  seem  exceptionally  competent  and  knowledgeable 
2 b.  seem  competent  and  knowledgeable 
1 c.  seem  adequate  in  their  knowledge 

0 d.  professor's  knowledge  seems  questionable 
Max  Points  Possible  = 3 

127.  The  professors  teaching  the  courses  that  I've  taken  in  engineering  and/or  technology 
teach  in  a way  that:  (Select  one) 

2 a.  motivates  me  to  want  to  learn  and  perform  in  those  classes  at  a very  high  level;  they  keep 
me  interested,  excited,  and  make  me  realize  that  I have  chosen  the  right  field  or  career 
track  for  me 

1 b.  keeps  me  interested  most  of  the  time  so  that  I perform  above  average  most  of  the  time 
0 c.  is  difficult  for  me  to  maintain  my  interest  in  the  courses;  it  is  often  difficult  to  remain 

interested  all  the  way  through  each  class;  I feel  I can  read  the  book  and  take  the  tests  and 
still  perform  well  enough  for  an  adequate  grade 
0 d.  truly  causes  me  to  be  less  motivated  to  perform,  making  it  almost  impossible  to  remain 
interested  in  the  courses  or  content  being  covered 
Max  Points  Possible  = 2 

128.  The  learning  environment  in  the  college  and  department  is  positive  in  the  following 

ways:  (Select  all  that  apply) 

1 a.  the  learning  environment  and  climate  are  positive 

1 b.  there  is  appropriate  technology,  computer  labs,  specialized  technology  related  to 
each  discipline 

1 c.  there  are  good  labs,  lab  equipment, 

1 d.  there  is  adequate  student  work  space  for  assignments,  projects,  group  meetings,  etc. 
1 e.  administrators  are  approachable  and  helpful  (e.g.,  the  department  chairs  (heads) 
and  dean) 

1 f.  faculty  are  available,  approachable,  professional,  and  helpful 
1 g.  department  and  college  staff  are  available,  professional,  and  helpful  in  solving 
problems  or  meeting  student  needs,  and  friendly 
1 h.  faculty  take  extra  time,  or  go  the  extra  mile,  and  are  available  to  support  and  assist 
students  in  solving  problems  or  meeting  their  needs 
1 i the  academic  advising  I have  received  is  of  high  quality  and  accurate 
1 j.  graduate  teaching  or  lab  assistants  seem  to  be  knowledgeable  and  competent 
Max  Points  Possible  = 10 
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129.  The  learning  environment  in  the  college  and  department  needs  to  improve  the 
following: 

(Select  all  that  apply) 

-1  a.  the  learning  environment  and  climate 

-1  b.  technology,  computer  labs,  specialized  technology  related  to  each  discipline 
-1  c.  labs  and  lab  equipment, 

-1  d.  student  work  space  for  assignments,  projects,  group  meetings,  etc. 

-1  e.  administrators  approachability  and  willingness  to  be  helpful  (e.g.,  the  department 
chairs  (heads)  and  dean) 

-1  f.  faculty  availability,  approachability,  professionalism,  and  willingness  to  be  helpful 
-1  g.  department  and  college  staff  are  availability,  professionalism,  and  helpfulness  in 
solving  problems  or  meeting  student  needs,  and  friendliness 
-1  h.  faculty  willingness  to  take  extra  time,  or  go  the  extra  mile,  and  be  available  to 
support  and  assist 

students  in  solving  problems  or  meeting  their  needs 
-1  i academic  advising 

-1  j.  knowledge  and  competence  of  graduate  teaching  or  lab  assistants 
Max  Points  Possible  = 0 

130.  Generally,  when  considering  course  quality,  the  courses  I've  taken  so  far  seem  to  have 
had  well  planned  content,  sound  academic  purpose,  appropriate  and  well  designed  lah 
activities,  and  excellent  execution  of  student  learning  activities  hy  the  professor  and/or 
grad  assistant. 

(Select  one) 

3 a.  strongly  agree 

2 b.  most  or  many  do 

1 c.  some  (less  than  half)  do 

0 d.  most  or  many  do  not 
Max  Points  Possible  = 3 

131.  The  courses  that  I've  taken  so  far  seem  to  have  been  well-structured  and  organized 
with  clear  learning  objectives  that  are  focused,  purposeful;  the  courses  have  had  well 
designed  and  developed  syllabi  that  clearly  explain  the  expectations  of  the  professor  for  the 
course  and  a schedule  or  timeline  provides  an  understanding  of  the  events,  due  dates,  and 
activities  for  the  semester. 

(Select  one) 

3 a.  strongly  agree 

2 b.  most  or  many  do 

1 c.  some  (less  than  half)  do 
0 d.  most  or  many  do  not 
Max  Points  Possible  = 3 
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For  Items  132-136,  consider  the  connections  between  course  syllabi,  assignments,  and 
schedule  for  all  the  courses  you  taken  to  date;  when  generalizing  across  ALL  the  courses 
you  have  taken  in  engineering  or  technology,  most  of  your  professors:  (Select  one  response 
for  each  132-134) 

132.  covered  the  course  content  specified  in  the  syllabus,  expanding  when  appropriate 

1 a.  yes 

0 b.  no 

Max  Points  Possible  = 1 

133.  adhered  to  the  assignments  specified  in  the  syllabus  and  didn't  add  anything 
significant 

1 a.  yes 

0 b.  no 

Max  Points  Possible  =1 

134.  progressed  through  the  course  according  to  the  schedule  plan  in  the  syllabus 

1 a.  yes 
0 b.  no 

Max  Points  Possible  = 1 

135.  In  most  of  my  courses,  the  tests  (or  other  methods  of  measuring  student  learning  such 
as  projects,  etc.,  papers,  research,  etc.)  are  directly  linked  and  connected  to  the  content 
described  in  the  syllabus.  (Select  one) 

2 a.  Yes,  most  of  the  time 

1 b.  Usually,  but  there  are  some  major  deviations  from  the  syllabi  across  courses 

0 c.  Less  than  half  of  the  time;  there  is  a lot  of  content  on  tests,  or  content  that  we  are 

required  to  know  and  use  for  projects,  etc.  that  was  not  specified  on  course  syllabi 
-Id.  There  has  often  been  a “dis-connect”  between  the  knowledge  and/or  skills  that  we 
were  tested  on  or  required  to  use  on  projects,  etc.  and  what  was  specified  on 
course  syllabi  across  the  courses  I have  taken 
Max  Points  Possible  = 2 

136.  In  most  of  my  courses,  the  tests  (or  other  methods  of  measuring  student  learning  such 
as  projects,  etc.,  papers,  research,  etc.)  are  directly  linked  and  connected  to  the  content 
covered  by  the  professors.  (Select  one) 

2 a.  Yes,  most  of  the  time 

1 b.  Usually,  but  there  have  been  some  major  deviations  by  the  professors  across 

courses 

0 c.  Less  than  half  of  the  time;  there  is  a lot  of  content  on  tests  or  content  that  we  were 
required  to  know  and  use  for  projects,  etc.  that  was  not  specified  on  course  syllabi 
or  covered  by  the  professors  or  assistants. 

-Id.  There  has  often  been  a “dis-connecf’  between  the  knowledge  and/or  skills  that  we  were 
tested  on  or  required  to  use  on  projects,  etc.  and  what  was  covered  by  the  professors  or 
assistants.  A lot  of  course  content  was  not  covered  by  the  professors  or  assistants. 

Max  Points  Possible  — 2 
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INTRODUCTION  TO  COURSE  ANALYSIS 

(See  Program  Description  in  Portfolio  Section  A. 5,  Course  Analysis  Data,  Reports,  and 
Professor  Examples  in  B.5.a,  1,  2,  3 and  B.5.b,  1,  2,  3;  also.  Worksheets  in  C.l) 

Jule  Dee  Scarborough,  Ph.D. 

Professors  each  chose  one  course  for  the  faculty  development  program  on  teaching  and 
learning.  The  selected  course  (taught  Fall  2005)  was  the  vehicle  for  learning.  Analyzing  a 
real  course,  instructional  practices,  student  assessments,  and  other  aspects  of  teaching  and 
learning  ensured  an  authentic  problem  based  learning  scenario  and  process.  The  2005 
course  was  changed  according  to  the  analyses  results  and  as  the  learning  process 
progressed  through  each  of  the  program  components.  The  course  was  then  re-developed 
during  particular  program  components,  once  again  ensuring  authentic  and  problem-based 
learning  through  performance  tasks.  The  re-developed  course  was  taught  during  the  Fall 
2006  research  semester  as  the  context  for  implementing  the  changes  and  the  experimental 
research  on  teaching  and  learning  by  each  professor.  Therefore,  professors  taught  a 
completely  re-developed  course  and  engaged  in  the  Scholarship  of  Teaching  through 
classroom  research  Fall  2006. 

The  faculty  development  program  engaged  professors  in  an  intense  analysis  of  their 
existing  2005  courses.  For  example,  they  analyzed  the  quality  of  the  course  content  using 
the  ABET/TAC/NAIT  outcomes;  they  considered  how  the  courses  were  taught,  the 
quality  of  their  teaching  strategies,  and  if  students  had  opportunities  to  use  or  expand 
their  individual  learning  styles.  The  professors  analyzed  their  tests  and  expanded  the 
student  assessment  tools  for  the  course  by  developing  performance  tasks  and  rubrics. 
They  considered  their  grading  strategies  and  philosophies,  the  quality  of  their  syllabus 
and  what  it  communicated,  and  much  more. 

The  learning  process  was  led  by  the  program  leader;  however,  it  was  also  led  by  the 
professors  themselves  in  that  they  used  self-assessment  through  a portfolio  assessment 
process.  This  provided  them  the  opportunity  to  critically  reflect  on  and  analyze  the 
“current  reality”  of  the  2005  courses  and  then  determine  what  needed  changing  regarding 
the  course,  teaching,  or  assessment  aspects.  The  program  process  involved  presentations, 
group  process  using  the  jigsawing  of  information,  a myriad  of  worksheets  with 
corresponding  reference  information  in  written  form,  group  review,  sharing,  and  critique, 
and  finally  redevelopment.  The  process  was  inquiry  based  and  guided  discovery  in 
action,  which  resulted  in  a performing- while-learning  process.  As  the  professors  analyzed 
their  courses  for  a variety  of  teaching  and  learning  factors,  they  were  also  learning  about 
teaching  and  learning.  Performing  the  analyses  was  performance-based  learning, 
resulting  in  educational  products,  and  the  learning  process  itself. 
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Each  professor  performed  the  analyses  listed  below  on  the  selected  2005  courses: 

1 . Knowledge  Content  (content  breakout  outline;  content  priorities;  content 
timeline) 

2.  Content  Sources 

3.  Embedded  NIU  General  Education  Goals 

4.  Content  Schedule 

5.  Teaching  Models 

6.  Teaching  Styles 

7.  Student  Eeaming  Styles 

8.  Bloom’s  Cognitive  Process  Dimension  (Traditional  or  Revised) 

9.  Bloom’s  Knowledge  Dimension 

10.  Dale’s  Cone  of  Eeaming 

1 1 . Eevel  of  Critical  Thinking 

12.  Teacher,  Knowledge,  Assessment,  or  Eearner-Centered? 

13.  Instmctional  Design  Gaps 

14.  Student  Eeaming  Outcomes  by 

a.  Bloom 

b.  Dale 

c.  Knowledge  Sources 

d.  Assessments  (Tests)  by 

i.  Bloom 

ii.  Dale 

e.  Test  Items  by 

i.  Bloom 

ii.  Dale 

f.  Other  assessments,  projects,  etc.  by 

i.  Bloom 

ii.  Dale 

15.  GAPS  Analysis  Summary 

a.  Outcomes  Summary 

b.  General  Education  Summary 

c.  Outcomes  by  Teaching  Models  Summary 

d.  Outcomes  by  Teaching  Styles  Summary 

e.  Outcomes  by  Student  Eeaming  Style  Opportunity  Summary 

f.  Outcomes  by  Bloom’s  Cognitive  Process  Dimension 

g.  Outcomes  by  Bloom’s  Knowledge  Dimension 

h.  Outcomes  by  Dale’s  Cone  of  Eeaming  Eevels 

i.  Outcomes  by  Critical  Thinking  Eevels 

16.  Test  Analyses 

a.  Item  quality 

i.  item  discrimination 

ii.  item  difficulty 
ill.  other 

(See  each  data  table  for  the  outcomes  of  this  process  in  Portfolio  Sections  B.0-B13  and 
worksheet  forms  in  Section  C) 


2 


The  professors  studied  the  results  of  the  2005  analyses,  made  decisions  about  what 
changes  would  benefit  student  learning,  and  engaged  in  redeveloping  the  courses  for  the 
2006  research  semester.  Course  re-development  involved  them  in 

1 . Student  Learning  Outcome  development  by  ABET/TAC/NAIT  standards  or 
outcomes 

2.  Identification  of  embedded  NIU’s  General  Education  Goals 

3.  Determination  of  Bloom’s  Knowledge  Dimension  for  each  student  learning 
outcome 

4.  Bloom’s  Cognitive  Process  Dimension  for  each  student  learning  outcome 

5.  Development  of  a course  calendar  identifying  for  each  week/course  day: 

a.  Teaching  Models 

b.  Teaching  Styles 

c.  Student  Beaming  Style  opportunities 

d.  Dale’s  Cone  of  Beaming  Bevels 

e.  Course  Content  Topics 

f.  Course  Babs  or  Activities 

g.  Course  Due  Dates 

6.  Test  Development 

a.  Table  of  Specifications 

b.  Item  Bank 

c.  Test  Assembly 

d.  Charting  student  learning  outcomes  by  tests  and  test  items 

7.  Performance  Assessments 

a.  Performance  Tasks 

b.  Corresponding  Rubrics 

8.  Multifaceted  Assessment  Plan 

a.  Identification  of  other  types  of  student  assessments 

c.  Charting  assessments  by  Bloom’s  Cognitive  Process  Dimensions 

d.  Charting  student  assessments 

6.  Teaching  Models  and  Instructional  Design  for  Course  Context 

a.  Choosing  additional  teaching  models 

b.  Designing  instruction  for  entire  course 

(See  Data  Tables  and  Teaching  Portfolios  for  these  products  B.0-B.13.) 

The  professors  tested  the  effectiveness  of  their  redeveloped  course,  teaching  and  learning 
strategies,  and  new  student  assessments  during  the  research  semester,  Pall  2006.  They 
also  engaged  in  experimental  research  on  a particular  teaching  and  learning  question. 

After  teaching  the  re-developed  courses,  the  professors  engaged  in  reflection  about  their 
teaching  practices  and  student  learning  during  the  2006  semester.  They  confirmed,  or 
identified  where  they  did  or  did  not  achieve,  the  planned 

1 . Teaching  Models  and  Teaching  Styles 

2.  Student  Beaming  Style  opportunities 

3.  Student  Beaming  Outcomes 

4.  Bloom’s  Cognitive  Process  Dimensions 

5.  Dale’s  Cone  of  Beaming  Bevels 
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In  addition,  they  engaged  in  critical  reflection  on  the  student  assessment  plan  and 
analyzed  the  effectiveness  of  the  new  tests  and  performance  assessments: 

1 . Midterm  exam  test  analysis  and  diagnostic  write  up 

2.  Final  exam  test  analysis  and  diagnostic  write  up 

3.  Performance  assessment/rubric  reflection  and  diagnostic  write  up 
(See  Portfolio  Section  B6) 

Important  Note:  In  the  Course  Analysis  Data  Sections,  most  of  the  worksheets  provide 
summaries  of  data  across  professors.  However,  several  worksheets  are  not  shown  with 
data,  as  the  GAPS  Analysis  Summary  provides  that  same  information.  The  analysis 
information  professors  generated  during  those  processes  was  then  summarized  in  the 
GAPS  Analysis  Summary.  The  information  in  those  two  analyses  was  so  specific  to  each 
professor,  it  would  be  impossible  to  present  a coherent  composite  for  1)  Content 
Schedule,  Models,  Styles,  Bloom’s,  etc.  and  2)  Instructional  Design  GAPs.  Thus,  that 
same  information  was  summarized  by  the  professors  in  the  overall  GAPS  Analysis 
Summary  composite.  All  other  worksheets  are  presented  as  complete  composites. 
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Discipline  Course  Outline  - lENG  475  - Decision  Analysis:  Regina  Rahn 


Course  Disciplinary 

Science(s) 

Mathematics 

Communication  Foundation/ 

Content 

Foundation  Required 

(Id.  Physics,  Chemistry,  Biology,  etc.) 

Foundation  Required 

Skills  Required 

I.  Linear  Value  Functions  and 

Decision  Making  Under  Uncertainty 

None 

A.  Algebra 

A.  Writing 

A.  Introduction 

B.  Calculus 

B.  Speaking 

B.  Decision  Trees 

C.  Probability  and  Statistics 

1 . Structure 


2.  Outcomes 

C.  Expected  Monetary  Value 

D.  Value  of  Perfect  and  Imperfect  Information 

E.  Bayes  Theorem 

1 . Overview  of  Probability 

2.  Joint,  Conditional,  and  Marginal  Probabilities 
E.  Conditional  Likelihood  Ratios 

G.  Sensitivity  Analysis 

II.  Utility  Theory  - Single  Attribute  Case 

A.  Motivation 

B.  Axioms 

C.  Degree  of  Risk  Aversion 

D.  Lotteries 

1 . Risk  Premium 

2.  Buying  Price 

3.  Selling  Price 

4.  Certainty  Equivalent 

5.  Insurance  Premium 

E.  Utility  Functions 

III.  Fault  Tree  Analysis 

A.  Event  Description 

B.  Construction  Methodology 

C.  Evaluation 

1 . Reliability  and  Failure  Probability  Relationships 

2.  Propagation  Through  the  Gates 

3.  Qualitative  Evaluation 

4.  Quantitative  Evaluation 

D.  Equivalent  Fault  Trees 

E.  Dual  Fault  Trees 

IV.  Utility  Theory  - Multi-Attribute  Case 

A.  The  General  Case 

B.  Additive  Functions 
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Table  B.5.a.l.l:  Content  Schedule  and 

1 Styles,  Models,  E 

doom's  Analysis  - lENG  475  Decision  Analysis:  Regina  Rahn 

Week 

Content  Topic: 

Factual,  Conceptual, 
Procedural,  Meta-cognitive 

Content 

Source 

Text,  etc. 

Teachi 

ng 

Style 

a-k;fpfd 

Learning 
Style  CE,  AE, 
AC,RO 

Teaching 

Model 

1-24  name 

Dale's  Cone 
Active  or  Passive 

Bloom's  Traditional:  Evaluation, 

Synthesis,  Analysis,  Application,  Comprehension 
, Knowledge 

Bloom's  Revised 

Create,  Evaluate,  Analyze,  Apply, 
Understand,  Remember 

Critical  Thinking 

(1-5,  1 is  low) 

Centered? 

Teacher,  Knowledge 
Assessment,  Learner 

1 

Introduction  - f,c 

LN,  text 
ch.  1,2 

A 

AR 

4 

HIGH  PASSIVE 

K 

R 

1 

Decision  tree  structure  & 
outcomes  - f,c,p 

LN,  text 
ch.  3,4 

A 

AR 

4 

HIGH  PASSIVE 

K 

R 

2 

2 

Probability  Overview  - 
f,P 

LN,  HO 

A 

AA,AR 

4 

HIGH  PASSIVE 

K 

R 

1 

Expected  Value  (EMV)  - 
f,c,p 

LN,  text 
ch.  4 

A 

AA,AR 

4 

HIGH  PASSIVE 

K 

R 

2 

3 

EVPI  & EVSI  - f,p 

LN,  CH. 
12,  paper 

A 

AR 

4 

HIGH  PASSIVE 

K 

R 

2 

Baye’s  Theorem  - f,p,c 

LN 

A 

AA,AR 

4,14 

HIGH  PASSIVE 

K 

R 

2 

4 

Intro  to  Sensitivity 
Analysis  - f,c,p,m 

LN,  text 
ch.  5 

A 

AA,AR 

4,14 

HIGH  PASSIVE 

K 

R 

2 

Case  Study  - p,c,m 

LN,  HO 

B 

CR,AR,AA 

12 

ACTIVE 

K,C,AP,AN 

R,U,AP,AN 

4 

5 

Continue  Case  Study  - 
c,p,m 

LN,  HO 

B 

CR,AR,AA 

7,12,13,14 

ACTIVE 

K,C,AP,AN 

R,U,AP,AN 

4 

Conditional  Likelihood 
Ratios  - f,c,p 

LN 

A 

AA,AR 

4,14 

HIGH  PASSIVE 

K 

R 

2 

6 

Intro  to  Utility  Theory  - 
f,c 

LN 

A 

AR 

4 

HIGH  PASSIVE 

K 

R 

1 

Motivation  and  Axioms  - 
f,c,m 

LN,  text 
ch.  13,14 

A 

AR 

4 

HIGH  PASSIVE 

K 

R 

2 

7 

Degrees  of  Risk  - f,c,p 

LN,  text 
ch.  13,14 

A 

AR 

4 

HIGH  PASSIVE 

K 

R 

2 

Lotteries  - f,c,p 

Ln 

A 

AA,AR 

4 

HIGH  PASSIVE 

K 

R 

2 

8 

Lotteries,  Calculations  & 
Examples  - c,p,m 

LN 

A 

AA,AR 

4,12 

LOW  ACTIVE 

K,C,AP 

R,U,AP 

3 

Two  Lotteries  at  a Time  - 
c,p,m 

LN 

A 

AR 

4 

HIGH  PASSIVE 

K 

R 

1 

9 

Single  Attribute  Utility 
Lunctions  - c,p 

LN,  text 
ch.  13 

A 

AA,AR 

4,12,14 

ACTIVE 

K,C,AP,AN 

R,U,AP,AN 

4 

Review  Session  - c,p,m 

Notes 

B 

CA,CR,AA 

12,13 

ACTIVE 

K,C,AP 

R,U,AP 

3 

10 

Mid-Term  Exam  - c,p,m 

Exam 

B 

CA,CR,AA 

AR 

12,13,14 

ACTIVE 

K,C,AP,AN 

R,U,AP,AN 

3 

Project  Discussion  - c,m 

Projects 

C 

CA,CR,AA 

5,14,15 

ACTIVE 

K,C,AP,AN,S,E 

R,U,AP,AN,E 

5 

11 

Intro  to  Eault  Trees  - f,c 

LN,  HO 

A 

AR 

4 

HIGH  PASSIVE 

K 

R 

1 

Discuss  Mid-Term  - 
c,p,m 

Go  over 
exam 

B 

AA,CR 

4,12 

LOW  ACTIVE 

K,C,AP,AN 

R,U,AP,AN 

4 

6 


12 

Evaluation  of  Fault  Trees 
(Qualitative)-  c,p,m 

LN,  HO 

A 

AR 

4 

HIGH  PASSIVE 

K 

R 

2 

Evaluation  of  Fault  Trees 
(Quantitative)-  c,p,m 

LN,  HO 

A 

AR 

4 

HIGH  PASSIVE 

K 

R 

2 

13 

Fault  Tree  Case  Study  - 
c,p,m 

LN,  HO 

B 

CR,AA,AR 

4,12,14 

ACTIVE 

K,C,AP,AN 

R,U,AP,AN 

4 

Equivalent  & Dual  Fault 
Trees  - f,c,p 

LN,  HO 

A 

AR 

4 

HIGH  PASSIVE 

K 

R 

2 

14 

Multi-Attribute  Utility 
Theory  (MAU)-  c,p 

LN,  paper, 
text  ch. 
15,16 

A 

AR 

4 

HIGH  PASSIVE 

K 

R 

2 

MAU  Calculations  and 
Examples  -c,p,m 

LN 

B 

CR,AR,AA 

4,12,14 

LOW  ACTIVE 

K,C 

K,C 

3 

15 

Review  of  Course  -c,p,m 

Review 

discussion 

A,B 

CR,AR,AA 

4,12,13 

LOW  ACTIVE 

K,C,AP,AN 

K,C,AP,AN 

4 

Project  Presentations  - 
c,p,m 

Projects 

B,C 

CA 

5 

ACTIVE 

K,C,AP,AN,S,E 

R,U,AP,AN,E 

5 

16 

Final  Exam  - c,p,m 

Final 

exam 

B 

CA,CR,AR, 

AA 

12,13,14 

ACTIVE 

K,C,AP,AN 

R,U,AP,AN 

4 

Note:  There  is  a practice  bv  some  professors  to  sive  the  final  exam  before  or  during  the  last  week  of  CLASS,  rather  than  the  FINAL  EXAM  WEEK.  This  essentially  means  that  you  are  "cheating"  the  students  out  of  one  day  of  content,  learning,  etc.  We  should  haye  15  weeks  of  learning, 
including  exam,  quizzes,  or  project  days,  but  to  make  the  final  week  of  class  the  FINAL  exam  week  is  unethical  by  NIU  standards,  regardless  of  who  does  it.  What  is  your  practice?  We  always  meet  on  the  day  of  the  final. 
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Table  B.5.a.l.2:  Instructional  Design  Gaps  Analysis  Table  - lENG  475  Decision  Analysis:  Regina  Rahn 


ABET/ 
NAIT 
Standard 
a-k  Eng 
A-Q  Tech 

NIU  General  Ed 
Goals  (embedded) 
a-I,  ii,  iii,  iv 
b-I,  ii,  iii,  iv 
c and  d 

Student  Learning  Objectives 
listed  on  syllabus 

Bloom/Dale 

Evaluation/Active 

Synthesis/Active 

Analysis/Active 

Application/Active 

Comprehension/P 

Knowledge/Passive 

Knowledge 

Sources 

Professor,  Text, 
Cases,  Speaker, 
References,  etc. 

Student 
Assessments 
listed  on 
syllabus 

Bloom/Dale 

Evaluation/Active 

Synth  es  is/A  ctive 

Analysis/Active 

Application/Active 

Comprehension/P 

Knowledge/Passive 

Test  Items 
or 

Proj  ects/Rubrics 

Bloom/Dale 

Evaluation 

Synthesis 

Analysis 

Application 

Comprehension 

Knowledge 

Performa 

nee 

IF  any; 
if  none, 
leave 
blank 

Bloom/Dale 

Evaluation/Active 

Synthesis/Active 

Analysis/Active 

Application/Active 

Comprehension/P 

Knowledge/Passive 

a,d,(h) 

To  study  the  meaning  and  application 
of  analytic  decision  making  techniques 
for  technical  decision  making  under 
uncertainty. 

Application, 

active 

See  content 
schedule 

HW,  Mid-term, 
Einal 

Application 

active 

HWl 

HW2  #3.25,4.14 
HW3  #13.24 
Midterm  #II 
Einal  #1, 4,5,6 

Application 

Application 

Application 

Application 

Application 

a,d,  (k) 

To  learn  a specific  set  of  analytical 
tools.  These  tools  are  applicable  to 
technical  decisions  that  must  be  made 
when  present  or  future  states  of  nature 
are  uncertain,  and  multiple  attributes 
are  considered.  Included  in  these 
methods  are  decision  tree  methodology 
and  utility  theory. 

Application 

HW,  Mid-term, 
Einal,  project 

Analysis,  some 

synthesis 

active 

HWl 

HW2  #3.25,4.14, 
5.9,5.12 
HW3  #13.27,28 
HW5#1,2 
Midterm  #I 
Einal  #3 

Analysis,  Syn 

Analysis 

Synthesis 

Analysis 

Analysis 

Analysis,  Syn 

Synthesis 

a,d, 

(b,k,i,j,f) 

To  become  aware  of  the  limitations  of 
the  models  for  rational  decision 
making. 

Evaluation, 

active 

Project 

Some 

evaluation 

active 

Project 

Evaluation 

Presentation 

and 

discussion 
of  projects 

a,d,c, 

To  allow  students  to  practice  the 
structuring  and  solution  of  complicated 
decision  problems. 

Evaluation, 

active 

HW,  Mid-term, 
final,  project 

Application, 
analysis,  some 
evaluation 
active 

HWl 

Midterm  #I 
Pinal  # 2,3,6 
Project 

Anal,  Syn 
Anal,  Syn 
Anal,  Syn 
Evaluation 
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Table  B.5.a.2.1:  Discipline  Course  Outline  (IENG370  - Reinaldo  Moraga) 


Course  Disciplinary 
Content 

Science(s) 

Foundation  Required 

Mathematics 
Foundation  Required 

Communication  Foundation/ 
Skills  Required 

I.  Overview 

II.  Linear  Programming  (LP) 

A.  LP  Assumptions 

B.  Model  Formulations 

1 . Graphical  Method 

2.  Classical  and  Large  Formulations 

C.  Simplex  Method 

1 . Standard  LP  Model  and  Transformations 

2.  Tabular  Form  and  Algorithm 

3.  Two  Phase  Method 

Not  directly  apply 

A.  Einear  Algebra 

1 . Systems  of  Equations 

2.  Operations  with  Matrices 

Quantitative  Reasoning 
Use  of  Technology  (Software) 
Communicate  in  Writing 
Interrelatedness  of  disciplines 

III.  Duality  Theory  and  Sensitivity  Analysis  (DSA) 

A.  Economic  Interpretation  of  duality 

B.  Primal-Dual  Relationships 

C.  Dual- Simplex  Method 

D.  Sensitivity  Analysis 

E.  Use  of  Software  EINDO 

Not  directly  apply 

Same 

Same 

IV.  Transportation  and  Assignment  Problems  (TAP) 

A.  The  Transportation  Problem 

B.  The  Assignment  Problem 

C.  The  Transshipment  Problem 

Not  directly  apply 

Same 

Same 

V.  Network  Optimization  Models  (NOP) 

A.  Terminology  of  Networks 

B.  The  Shortest-Path  Problem 

C.  The  Minimum  Spanning  Tree  Problem 

D.  The  Maximum  Plow  Problem 

E.  The  Minimum  Cost  Plow  Problem 

Not  directly  apply 

Same 

Same 

VI.  Integer  Programming  (IP) 

A.  Assumptions 

B.  Pormulations 

C.  Branch  and  Bound  Algorithm 

Not  directly  apply 

Same 

Same 

VII.  Dynamic  Programming  (DP) 

A.  Characteristics  of  Dynamic  Programming 

B.  Deterministic  Dynamic  Programming 

Not  directly  apply 

Same 

Same 
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Table  B.5.a.2.2:  Content  Schedule  and  Styles,  Models,  Bloom's  Analysis 


Week 

Content  Topic: 

Factual,  Conceptual, 
Procedural,  Meta-cognitive 

Content 

Source 

Text,  etc. 

Teaching 

Style 

a-k;fpfd 

Learning 
Style  CE, 
AE,  AC,  RO 

Teaching 

Model 

1-24  name 

Dale's  Cone 
Active  or  Passive 

Bloom's  Traditional:  Evaluation, 

Synthesis,  Analysis,  Application, 
Comprehension , Knowledge 

Bloom's  Revised 

Create,  Evaluate,  Analyze,  Apply, 
Understand,  Remember 

Critical  Thinking 

Centered? 

Teacher,  Knowledge 
Assessment,  Learner 

1 

Syllabus,  Blackboard 

Overview 

Inst. 

a,b 

Knowledge,  Comprehension 

Understand,  Apply 

2 

2 

LP  - Assumptions 

Chi 

a,b 

AE,  AC 

4Lecture, 

7CoopLrni 

ngl4Deduc 

live 

Passive 

Knowledge,  Comprehension 

Understand,  Apply 

2 

LP  - Models 

Ch2&3 

a,b 

AE,  AC 

same 

Passive/Active 

Knowledge,  Comprehension, 
Application 

Understand,  Apply 

2 

3 

Holliday 

Knowledge,  Comprehension, 
Application 

Understand,  Apply 

LP  - Models 

Ch3 

a,b 

AE,  AC 

same 

Passive/Active 

Knowledge,  Comprehension, 
Application 

Understand,  Apply 

2 

4 

LP  - Simplex 
Method 

Ch4 

a,b 

AE,  AC 

same 

Passive 

Knowledge,  Comprehension, 
Application 

Understand,  Apply 

3 

LP  - Simplex 
Method 

Ch4 

a,b 

AE,  AC 

same 

Passive 

Knowledge,  Comprehension, 
Application 

Understand,  Apply 

3 

5 

LP  - Simplex 
Method 

Ch4 

a,b 

AE,  AC 

same 

Passive/Active 

Knowledge,  Comprehension, 
Application 

Understand,  Apply 

4 

LP  - Simplex 
Method 

Ch4 

a,b 

AE,  AC 

same 

Passive/Active 

Knowledge,  Comprehension, 
Application 

Understand,  Apply 

4 

6 

DSA  - El,  P/D 
Relationship 

Ch6 

a,b 

AE,  AC 

same 

Passive 

Analysis 

Understand,  Apply,  Analyze 

4 

DSA  - D/S  Method 

Ch6 

a,b 

AE,  AC 

same 

Passive 

Analysis 

Understand,  Apply,  Analyze 

4 

7 

DSA  - S.  Analysis 

Ch6 

a,b 

AE,  AC 

same 

Passive/Active 

Analysis 

Understand,  Apply,  Analyze 

4 

DSA  - Software 

a,b 

AE,  AC 

same 

Passive/Active 

Knowledge,  Comprehension 

Apply 

4 

8 

In-class  Exam 

Mini-case  in  class 

a,b 

AE,  AC 

same 

Active 

Comprehension,  Application, 
Evaluation 

Analyze,  Evaluate 

5 

9 

TAP  - Transportation 

Ch8 

a,b 

AE,  AC 

same 

Passive 

Application,  Evaluation 

Understand,  Apply,  Analyze 

4 

TAP  - Transportation 

Ch8 

a,b 

AE,  AC 

same 

Passive 

Application,  Evaluation 

Understand,  Apply,  Analyze 

4 

10 

TAP  - Assignment 

Ch8 

a,b 

AE,  AC 

same 

Passive 

Application,  Evaluation 

Understand,  Apply,  Analyze 

4 

TAP  - Transshipment 

Ch8 

a,b 

AE,  AC 

same 

Passive 

Application,  Evaluation 

Understand,  Apply,  Analyze 

4 

11 

NOP  - Terminology, 
SP  problem 

Ch9 

a,b 

AE,  AC 

same 

Passive 

Application,  Evaluation 

Understand,  Apply,  Analyze 

4 

NOP  - STP,  MEP 

Ch9 

a,b 

AE,  AC 

same 

Passive 

Application,  Evaluation 

Understand,  Apply,  Analyze 

4 

12 

NOP  - MCEP 

Ch9 

a,b 

AE,  AC 

same 

Passive 

Application,  Evaluation 

Understand,  Apply,  Analyze 

4 

10 


IP  - Assmptn, 
Formulations 

Chll 

a,b 

AE,  AC 

same 

Passiye/Active 

Knowledge,  Comprehension, 
Application 

Understand,  Apply 

3 

13 

IP  - BB  algorithms 

Chll 

a,b 

AE,  AC 

same 

Passiye 

Knowledge,  Comprehension, 
Application 

Understand,  Apply 

3 

DP  - Characteristics 

ChlO 

a,b 

AE,  AC 

same 

Passiye 

Knowledge,  Comprehension, 
Application 

Understand,  Apply 

3 

14 

DP  - Example 

ChlO 

a,b 

AE,  AC 

same 

Passiye 

Knowledge,  Comprehension, 
Application 

Understand,  Apply 

3 

Holliday 

15 

Project  Presentations 

c,d 

Synthesis,  Eyaluation 

Analyze,  Create 

5 

Project  Presentations 

c,d 

Synthesis,  Eyaluation 

Analyze,  Create 

5 

16 

Take-Home  Final 
Exam  Due 

Note:  There  is  a practice  by  some  professors  to  give  the  final  exam  before  or  during  the  last  week  of  CLASS,  rather  than  the  FINAL  EXAM  WEEK.  This  essentially  means  that  you  are  "cheating"  the  students  out  of  one  day  of  content,  learning,  etc.  We 
should  haye  15  weeks  of  learning,  including  exam,  quizzes,  or  project  days,  but  to  make  the  final  week  of  class  the  EINAL  exam  week  is  unethical  by  NIU  standards,  regardless  of  who  does  it.  What  is  your  practice? 
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Gaps  Analysis  Table 


Bloom/Dale 

Evaluation/Active 

Synthesis/Active 

Analysis/Active 

Application/Active 

Comprehension/P 

Knowledge/Passive 

Test  Items 
or 

Projects/Rubric 

s 

Bloom/Dale 

Evaluation 

Synthesis 

Analysis 

Application 

Comprehension 

Knowledge 

Performance 
IF  any; 
if  none, 
leave 
blank 

Bloom/Dale 

Evaluation/Active 

Synthesis/Active 

Analysis/Active 

Application/Active 

Comprehension/P 

Knowledge/Passive 

Synthesis, 

Analysis, 

Application 

Comprehension 

Problem  solving 
Quiz  1 . Eormulation 
Quiz  2.  Simplex  Me- 
thod 

Quiz  3.  Sensitivity 
Analysis 

Quiz  4.  Transporta- 
tion 

Quiz  5.  Transporta- 
tion 

Quiz  6.  Networks 

Synthesis/Active 

Application/Active 

Analysis/Active 

Application/Active 

Application/Active 

Application/Active 

Exercises  from 
Chapters 

Hmwl.  2.2-2,  2.3-1 
Hmw2.  3.1-9,  3.4-14 
Hmw3.  4.1-8,  4.3-4, 
4.7-4. 

Comprehen/Pasive 

Synthesis/Active 

Application/Active 

Case  study  4.1 

Ev  alUation/Active 

Evaluation 

Synthesis 

Midterm  (F'  half): 

In  class  (.3): 

PI.  Eormulation  3.4- 
12 

P2.  Solving  Simplex 
Method 
P3.  14  Multiple 
choice  questions 

Synthesis/Active 

Application/Active 

Comprehension/Passiv 

e 

Synthesis 

Application 

Comprehension 

TkHm  (.7): 

PI.  Eormulation 
P2.  Solving  Simplex 
Method 

Synthesis/Active 

Application/Active 

Application 

Comprehension 


Final  Exam: 
PI  9.3-6 


Application/Active 


(30%). 


An  optional  term 
paper  to  help  those 
interested  in  getting  a 
better  grade. 


Table  B.5.a.3.1:  Discipline  Course  Outline 

Course  Disciplinary  Science(s) 

Content  Foundation  Required 

(Id.  Physics,  Chemistry,  Biology,  etc.) 

I.  Unit 

A.  Review  of  Dynamics  A.  Physics,  Chemistry,  Bio? 

1. 

1.  Particle  Kinematics  1.  Dynamics  (Particle  and  Rigid  Body 

2.  Rigid  Body  Kinematics  FBD,  MAD,  Kinematics,  Kinetics) 

a)  Relative  Velocity 

b)  Relative  Acceleration 

3.  Particle  and  Rigid  Body  Kinetics 

a)  Force-Mass-Acceleration 

b)  Work-Energy 

c)  Impulse-Momentum 

4.  Explain  how  to  decide  what 
Method  to  use  by  identifying 
variables 


B.  Basic  Concepts  and  Classification  of  Vibration 

1 . Elementary  parts 

2.  Degrees  of  Ereedom 

3.  Types  of  Vibration 

C.  Spring,  Mass,  and  Damping  Elements 

1 . Equivalent  spring 

2.  Equivalent  mass 

D.  Harmonic  motion  & Analysis 
of  periodic  function 

1 . Definitions 

2.  Eourier  Series 

3.  Eab  on  Eourier  Series  3.  Ability  to  use  spectrum  analyzer 


C.  Dynamics 

1 . Potential  energy  of  spring 

and  gravity 

2.  Kinetic  Energy  of  mass 


B.  Dynamics 

1 . Coordinate  Systems 


Mathematics 
Foundation  Required 


Communication  Foundation/ 
Skills  Required 


A.  Calculus 

1.  Vector  Algebra 

2.  Integration 

3.  Differentiation 


D.  Mathematics 

1 . Develop  equations  of  graphs  shown 

2.  Integration  over  a domain 

3.  Ability  to  use  Matlab 

Writing 
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II. 


Unit 

A.  Free  Vibration  of  Undamped  and  Damped  Vibration  A.  Dynamics 


A.  Mathematics 


1.  Equation  of  motion  from  FBD&MAD  I.  Force-Mass-Acceleration  1 . (Solution  of  order  ordinary 

2.  Equation  of  motion  using  other  methods  2.  Conservation  of  energy  differential  equations) 

3.  Solution  of  equation  of  motion 

4.  Examples  (how  complicated  models  are  simplified) 

5.  Reverse  analysis  (moment  of  inertia  of  rigid  body 
from  compound  pendulum  analysis) 

6.  Experiment  on  determination  of  natural  freq.  of  6.  Ability  to  use  EET  Analyzer 

SDOE  Some  digital  signal  processing  Writing 

issues 

B.  Dynamics  B.  Mathematics 

1.  Eorce-Mass- Acceleration  1.  (Solution  of  2"“^  order  ordinary 

homogenous  differential  equations) 


A.  Dynamics  A.  Mathematics 

1.  Eorce-Mass-Acceleration  1.  (Complementary  Solution 

and  particular  integral  of 
non-homogenous  diff  eqns) 

2.  Response  of  a damped  system  2.  Same  2.  Same 

3.  Quality  factor  and  bandwidth 

3.  Base  Excitation 

4.  Transmissibility  and  Vibration  Isolation  3.  Same  4.  Same 

4.  Rotating  unbalance  4.  Same  4.  Same 

IV.  Unit 

A.  Eorced  vibration  of  SDOE  subjected  to  general  A.  solution  of  diff.  eqn 

Vibration  conditions 

1 . Response  under  periodic  force 

2.  Response  under  transient  force 


B.  Eree  Vibration  of  Damped  System 

1.  Viscous  Damping 

2.  Damping  ratio,  under, critical, & over  damped 

3.  Eog  decrement,  energy  dissipated 

4.  Other  types  of  damping 

III.  Unit 

A.  Harmonically  Excited  Vibration  of  SDOE 
1 . Response  of  an  undamped  system 
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V.  Unit 

A.  Two  Degree  of  Freedom  Systems  A.  Dynamics  (Force-Mass-Acceleration)  A.  Matrix  algebra 

B.  Concept  of  natural  frequencies  and  associated 
Mode  shapes 

C.  General  eigenvalue  problem  C.  Matlab  solutions 

D.  Determination  of  natural  frequencies  D.  Understand  how  FRF  D.  Complex  matrix  algebra 

And  mode  shapes  is  used  to  obtain  modal  properties 

E.  Steady  State  response  due  to 
Harmonic  excitation  - Direct  method 

VI  Unit 

A.  Multiple  Degree  of  Freedom  Systems 

B.  Eigenvalue  Problem 

C.  Orthogonal  property  and  mass  normalization 
of  modes 

D.  Vibration  of  undamped  systems  using  modal 
Analysis 

E.  Eab  on  MDOE 

VI.  Unit 

A.  Vibration  Isolation  Same 

B . T uned  Ab  sorber 


Writing 
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Table  B.5.a.3.2:  Content  Schedule  and  Styles,  Models,  Bloom's  Analysis 


Week 

Content  Topic: 

Factual,  Conceptual, 
Procedural,  Meta-cognitive 

Content 

Source 

Text,  etc. 

Teaching 

Style 

a-k;fpfd 

Learning 
Style  CE,  AE, 
AC,RO 

Teaching  Model 

1-24  name 

Dale's  Cone 
Active  or  Passive 

Bloom's  Traditional 

Evaluation,  Synthesis, 
Analysis,  Application, 
Comprehension  ,Knowledge 

Bloom's  Revised 

Create,  Evaluate,  Analyze, 
Apply, 

Under  stand,  Remember 

Critical 
Thinking 
High  (H), 
Medium  (M), 
Low(L) 

Centered? 

Teacher,  Knowledge 
Assessment,  Learner 

1 

Introduction 

Prof. 

a,b 

AR 

Fecture, 

Concept 

formulation 

Conceptualization 

passive 

knowledge 

PK 

H 

Review  of 
Dynamics 

Prof, 
Dyn  text 

a,b,d,f 

AA 

Advance 

organizer 

Passive,  active 

Comprehension, 

MK 

H 

Basic  Concepts 

Prof, 

Text 

A,b 

AR,aa 

Fecture 

passive 

knowledge 

CK 

M 

2 

Equivalent  Spring, 
mass 

Text 

A,b,d 

AR,AA 

Simulations 

passive 

application 

PK 

M 

Harmonic  Motion 

Text 

A,b 

AR 

Fecture 

passive 

comprehension 

CK 

F 

Fourier  series 

Text 

A,b,f 

AR 

Fecture 

passive 

comprehension 

PK 

M 

3 

Free  Vib  of  SDOF 
Undamped 
Translational  syst, 
Fab  on  Fourier 
series 

Text 

Manuals, 

Tutorial 

A,b,f,c 

CR,CA,AR 

Fecture 

Cooperative 

Feaming 

Passive,  active 

comprehension 

PK 

H 

Free  Vib  of  SDOF 
Undamped 
Rotational  syst 

Text, 

prof 

A,b 

CR 

Fecture 

passive 

knowledge 

PK 

M 

Examples 

prof 

A,b,f 

AA 

Conceptualization 

passive 

application 

PK 

M 

4 

Free  Vib  of  damped 
syst.,  damping  ratio. 

text 

A,b 

AA,AR 

Fecture 

passive 

comprehension 

CK 

H 

Fog  Decrement, 
Energy  dissipated 

text 

A,b 

AA 

Fecture 

passive 

knowledge 

PK 

M 

Other  types  of 
damping 

Text 

prof 

A,b 

CR 

Fecture 

passive 

knowledge 

PK 

M 

5 

Response  of 
undamped  syst. 
excited 
harmonically. 

Text 

Manuals 

A,b,c,d,f 

AR,AA 

Fecture, 

Cooperative 

learning 

Passive,  active 

comprehension 

PK 

H 
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Lab  SDOF  free  vib. 

Response  of 
damped  syst. 
excited 
harmonically 

Text 

A,b 

AR,AA 

Fecture 

passive 

knowledge 

CK 

H 

Quality  factor  & 
bandwidth 

Text 

Prof 

A,b 

AA 

Fecture 

passive 

comprehension 

PK 

M 

6 

Exam#l 

active 

comprehension 

PK 

H 

Base  excitation 

Text 

A,b 

AR 

Fecture 

passive 

knowledge 

CK 

M 

Transmissib, 

Isolation 

Text, 

Prof. 

A,b 

AR,AA 

Fecture,  concept 

Presentation, 

conceptualization 

passive 

comprehension 

PK 

H 

7 

Rotating  Unbalance 

Text 

A,b 

AR 

Fecture 

passive 

knowledge 

CK 

M 

Review 

Passive,active 

application 

MK 

M 

Response  of  SDOF 
sub.  to  Periodic 
force 

Text 

A,b 

AR 

Fecture 

passive 

comprehension 

CK 

M 

8 

Response  of  SDOF 
sub.  to  transient 
force 

Text, 

Prof 

A,b 

AR 

Fecture 

passive 

comprehension 

CK 

H 

Matlab  solutions 

Prof. 

A,b,f 

AA 

Fecture, 

passive 

comprehension 

PK 

H 

Free  vib  of 
undamped  2DOF 

Text 

A,b 

AR 

Fecture 

passive 

knowledge 

CK 

M 

9 

Natural  freq.  & 
mode  shapes 

Text, 

Prof 

A,b 

AR 

Fecture, 

conceptualization 

passive 

comprehension 

PK 

H 

Coordinate 

coupling 

Text 

A,b 

AR,AA 

Fecture 

passive 

application 

PK 

M 

Review 

A,b,f 

Passive,active 

comprehension 

MK 

M 

10 

Exam#2 

active 

application 

MK 

H 

Steady  state 
response  due  to 
harmonic  excitation 
- direct  method 

Text 

A,b 

AR 

Fecture 

passive 

knowledge 

PK 

M 

Multiple  DOF 

Text 

A,b 

AR 

Fecture 

passive 

comprehension 

PK 

M 

11 

Eigenvalue  prob 

Text,Prof 

A,b 

AR,AA 

Fecture 

passive 

knowledge 

PK 

M 
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Orthogonality  of 
modes  and  mass 
normalization 

Text 

Prof 

A,b 

AR 

Lecture, 

conceptualization 

passive 

comprehension 

CK 

M 

Vibration  of 
undamped  syst. 
using 

modal  analysis 

Text 

A,b 

AR,AA,CR 

Lecture 

passive 

application 

PK 

H 

12 

Forced  vib.  of 
damped  syst,  Lab 
on  free  vib.  of 
MDOF 

Text 

Prof 

A,b,c,d,f 

AA,AR 

Lecture, 

cooperative 

learning 

passive,  active 

application 

PK 

H 

Matlab  solutions 

Prof 

A,b,f 

AA 

passive 

comprehension 

PK 

H 

Review 

passive,active 

application 

MK 

M 

13 

NVH  issues 

Prof. 

A,b,f 

CR 

Lecture,  Advance 
Organizer 

passive,  active 

application 

MK 

M 

Single  plane 
unbalancing 

Text 

A,b 

AA 

Lecture 

passive 

knowledge 

CK 

M 

Vibration  Isolation 

Text 

A,b 

AR 

Lecture 

passive 

comprehension 

PK 

M 

14 

Designing  of 
systems 

Prof 

A,b,c,d,e,f 

CA 

Lecture, 

reciprocal 

learning. 

Cooperative 

learning 

active 

synthesis 

MK 

H 

Tuned  undamped 
absorber 

Text 

A,b 

AA 

Lecture 

passive 

comprehension 

CK 

M 

Lab  on  balancing 

Text 

A,b 

AR 

Cooperative 

learning 

active 

comprehension 

CK 

M 

15 

Industrial  issues 

Prof. 

A,b 

CR 

Lecture 

passive 

analysis 

MK 

M 

Review 

passive,active 

application 

M 

Review 

passive,  active 

comprehension 

M 

16 

Final  Exam  or 
Project 

active 

comprehension 

MK 

H 

Note:  There  is  a practice  bv  some  professors  to  give  the  final  exam  before  or  during  the  last  week  of  CLASS,  rather  than  the  FINAL  EXAM  WEEK.  This  essentially  means  that  vou  are  "cheating"  the  students  out  of  one  dav  of  content,  learning,  etc.  We  should 
have  15  weeks  of  learning,  including  exam,  quizzes,  or  project  days,  but  to  make  the  final  week  of  class  the  FINAL  exam  week  is  unethical  by  NIU  standards,  regardless  of  who  does  it.  What  is  your  practice?  I GIVE  FINAL  EXAM  ONLY  AT  TIME 
SCHEDULED  IN  CATALOG  (AND  NOT  DURING  FINAL  WEEK  OF  CLASS) 

19 


Table  B.5.a.3.3:  Instructional  Design  Gaps  Analysis  Table 


ABET/ 
NAIT 
Standard 
a-k  Eng 
A-Q  Tech 

NIU  General  Ed 
Goals  (embedded) 
a-I,  ii,  iii,  iv 
b-I,  ii,  iii,  iv 
c and  d 

Student  Learning 
Objectives 
listed  on  syllabus 

Bloom/Dale 

Evaluation/Active 

Synthesis/Active 

Analysis/Active 

Application/Active 

Comprehension/P 

Knowledge/Passive 

Knowledge 

Sources 

Professor,  Text, 
Cases,  Speaker, 
References,  etc. 

Student 
Assessments 
listed  on 
syllabus 

Bloom/Dale 

Evaluation/Active 

Synthesis/Active 

Analysis/Active 

Application/Active 

Comprehension/P 

Knowledge/Passive 

Test 

Items 

or 

Projects/ 

Rubrics 

Bloom/Dale 

Evaluation 

Synthesis 

Analysis 

Application 

Comprehension 

Knowledge 

Performance 
IF  any; 
if  none, 
leave 
blank 

Bloom/Dale 

Evaluation/Active 

Synthesis/Active 

Analysis/Active 

Application/Active 

Comprehension/P 

Knowledge/Passive 

A,  E,  I,  J 

To  learn  basic  tbeories  behind 
vibrating  mass  or  equipment 

Application 

Professor, 

text 

Hw,  tests 

Analysis 

Exam2: 
Probl,  2,3 
Pinal 
Exam: 
Prob2,3,4 

K 

To  familiarize  with  various 
Equipment  and  software  used 
to  run  equipment 

Comprehen. 

Professor, 

TA, 

References 

lab 

Comprehen. 

B,  D 

To  perform  experiment  to 
verify  the  theories 

Application 

Professor, 

TA 

lab,  test 

Application 

Pinal 

Exam: 

Probl 

G 

To  learn  how  to  write  a 
laboratory  report 

Knowledge 

Professor, 

TA 

lab 

Knowledge 

C,E,K 

To  apply  major  commercially 
available  mathematical  and 
engineering  software.  Matlab 
solutions  of  differential 
equations  in  vibration  related 
problems 

Application 

Professor 

hw 

Application 

A,C,E,H,J,  and 
K 

To  design  structures  where 
failure  from  vibration  is 
prevented 

Synthesis 

Professor, 

references 

Hw,  tests 

Application 

Exam2: 

Prob2, 

Pinalexam 

Prob2,4 
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INSTRUCTIONAL  GAPS  ANALYSIS  SUMMARY  (FALL  2005  AND  FALL  2006) 

(See  Professor  Examples,  B.5.b,  1,2,3;  also.  Program  in  A. 5 and  Worksheets,  Section  C ) 

Jule  Dee  Scarborough,  Ph.D. 

Student  Learning  Statements  (Outcomes) 

In  the  initial  analysis  of  the  Fall  2005  courses  - where  we  began,  professors  used  their 
existing  course  syllabi.  Although  as  a college  we  had  improved  our  student  learning 
statements  during  the  accreditation  process,  they  remained  rather  unorganized  and  weak 
in  content  and  appropriate  expression.  The  learning  statements  were  expressed  in  mixed 
modes  across  syllabi.  Some  learning  statements  were  written  as  course  objectives;  others 
were  written  as  student  learning  objectives;  yet  others  were  written  as  more  outcome- 
oriented  statements.  However,  in  generalizing,  many  and  sometimes  most  of  the  student 
learning  statement  formats  across  syllabi  were  not  active,  clear,  measurable,  or  clearly 
outcomes-oriented,  where  the  professor  and  student  could  ascertain  exactly  what  was 
expected  and  would  be  measured  and/or  determine  the  culminating  grade.  Three 
professors  expressed  the  learning  statements  in  a way  where  students  could  see  that  there 
was  a relationship  between  student  learning  outcomes  and  the  ABET  or  NAIT  outcomes, 
but  if  the  ABET  or  NAIT  outcomes  were  identified  by  a letter  and  not  stated,  then  the 
relationship  was  not  clear,  nor  were  students  about  to  review  the  accreditation  outcomes 
for  their  own  information.  Two  professors  expressed  the  statements  more  clearly,  with 
written  statements  for  both  the  national  standards  and  the  learning  outcomes  for  the 
course.  The  other  five  professors  did  not  show  the  national  statements  in  narrative  but 
rather  identified  them  by  letter  or  number,  regarding  the  level  of  coverage  and  depth  of 
relationship.  This  had  little  meaning  for  students  and  did  not  make  it  easy  for  the 
professors  to  clearly  be  assured  of  direct  links  and  relationships.  Generally,  the 
statements  did  link  to  the  ABET  or  NAIT  standards  or  outcomes,  but  often  not  clearly  or 
strongly.  It  would  have  been  difficult  to  determine  a direct  link,  especially  in  light  of  the 
student  learning  assessments  being  used  for  the  2005  course.  Therefore,  we  examined  the 
2005  syllabi  and  course  content  related  to  the  standards  as  well  as  we  could,  with  the 
understanding  that  the  student  learning  outcomes  to  be  redeveloped  would  better  and 
more  clearly  and  directly  link  to  the  national  standards  and  assessments  - a two-way  link 
revealing  the  critical  knowledge  and  skill  connections. 

Below  are  two  charts  that  broadly  identify  the  standards  addressed  in  the  Fall  2005 
courses,  according  to  the  content  and  syllabus  analysis  by  each  professor  of  his/her 
course.  The  data  are  presented  (in  black)  as  collapsed  across  either  all  engineering 
professors  or  engineering  technology  professors  as  a broad  viewpoint.  The  Fall  2006 
courses  are  presented  in  red,  and  although  there  are  minor  differences  in  the  number  of 
standards  addressed,  there  is  a great  and  very  significant  difference  in  the  quality  of  the 
learning  statements  and  their  direct  links  to  both  the  national  standards  and  the  learning 
assessments.  The  tables  also  reflect  the  number  of  learning  outcomes  for  each  standard  by 
professor,  2005-2006  when  possible.  For  the  2006  courses,  the  professors  not  only  have 
improved  wording  and  expression,  but  the  knowledge  and  skill  connections  are  much 
stronger;  in  addition,  the  outcome  statements  are  improved  because  they  are  broken  out 
into  primary,  second,  and  third  level  statements.  The  quality  is  improved  not  only 
because  of  better  wording,  but  also  because  they  now  better  understand  the  difference 
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between  complex  statements,  where  there  is  a cluster  of  outcomes  inherent  to  a single 
primary  outcome  statement.  Thus  the  course  content  or  the  knowledge  and  skills  to  be 
taught  became  more  obvious  in  the  inherent  breakouts  of  second  and  sometimes  third 
level  outcomes.  This  provided  insight  and  assisted  the  professors  in  understanding  what 
underlying  or  inherent  knowledge  and  skills  were  required  for  a complex  cluster  of 
difficult  primary  learning  outcomes  - in  other  words,  the  knowledge  and  skills  inherent  to 
a single  complex  primary  outcome.  Therefore,  readers  may  be  amazed  at  the  number  of 
changes  that  resulted. 

Usually,  the  primary  statements  would  be  used  on  syllabi  or  other  reporting  documents, 
but  the  analysis  and  breakout  of  second  or  third  level  learning  statements  provided  a great 
learning  experience  for  professors  and  led  them  to  design  and  then  engage  students  in 
more  intentional,  thoughtful,  and  higher  quality  learning  experiences.  This  analysis  and 
process  can  lead  to  more  astute  teaching  and  student  learning,  student  assessments, 
instructional  choices,  learning  process  decisions,  and  more.  Remember,  each  course  is 
not  required  to  address  every  national  standard  or  outcome,  but  instead  the  standards  or 
outcomes  of  focus  selected  should  be  addressed  well.  It  is  important  that  they  understand 
individual  course  versus  program  requirements,  that  there  is  a cumulative  effect  across 
courses  for  the  entire  program,  and  that  the  overall  program  is  required  to  address  all 
national  standards  or  outcomes,  not  any  single  course;  therefore,  many  standards  will  be 
addressed  across  multiple  courses.  However,  particular  standards  may  be  addressed  in 
only  one  or  two  courses  across  the  program,  depending  on  content,  depth,  program  level, 
(e.g.,  introductory  or  capstone  course).  Professors  sometimes  mistakenly  strive  to  address 
all  or  too  many  outcomes;  thus  the  course  content  can  become  weak  or  superficial. 
Finally,  when  identifying  the  objectives  or  outcomes  listed  below,  an  * is  used  where  one 
objective  or  outcome  covers  more  than  one  ABET  outcome  or  NAIT  standard  or  where 
there  is  a greater  total  of  “Is”  than  the  total  in  the  number  in  parentheses  (4).  The 
determining  factor  is  the  level  of  coverage  of  content. 

Regarding  outcomes,  it  is  important  to  note  that  the  professors  analyzed  the  engineering 
or  technology  course  content  for  embedded  NIU  General  Education  Goals.  This  analysis 
led  them  to  more  deeply  understand  why  students  fail  to  perform  well  in  their  courses  if 
they  do  not  come  to  the  course  with  the  appropriate  general  education  knowledge  and 
skills  that  are  the  underlying  foundation  for  the  engineering  and  technology  content.  The 
program  leader  revealed  the  strong  relationships  between  NIU  General  Education  Goals 
(outcomes)  and  the  ABET  and  NAIT  standards  or  outcomes  by  aligning  and  inserting 
them  into  a worksheet.  That  made  it  much  easier  and  more  efficient  for  the  professors  to 
see  the  direct  relationships,  to  consider  the  importance  of  acknowledging  the  embedded 
general  education  goals/outcomes  as  part  of  their  course  content,  and  to  realize  that  even 
though  they  are  teaching  engineering  or  technology  courses,  they  are  actually 
concurrently  continuing,  extending,  expanding,  and  deepening  the  learning  of  general 
education  content  in  the  context  of  engineering  and  technology.  This  was  extremely 
important.  Our  professors  intuitively  knew  this  but  had  never  “studied”  the  connections, 
mapped  the  connections,  or  included  the  general  educational  goals  aligned  beside  their 
engineering  or  technology  outcomes.  They  had  also  never  thought  of  themselves  as 
continuing  the  learning  of  the  general  education  knowledge  and  skills  in  engineering  and 
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technology  content.  They  considered  the  general  education  math,  science,  and 
communication  knowledge  and  skills  as  prerequisites  and  only  dealt  with  them  when 
students  did  not  have  the  knowledge  or  skills  needed  to  perform  on  the  engineering  and 
technology  content.  Now  the  professors  understand  that  they  actually  continue  the 
learning  of  the  mathematics,  science,  communication,  etc.  content  in  the  engineering  and 
technology  context.  The  chart  below  reflects  the  2005  course  in  black  and  the  changes  for 
the  2006  course  in  red.  The  professors  improved  the  outcomes  and  connections  and  are 
committed  to  greater  depth  of  change  for  the  future.  This  was  a very  successful  program 
component,  resulting  in  significant  learning  and  change. 

Assumptions 

Beware  of  assumptions  when  scanning  the  chart  below  and  noting  that  one  or  more 
course  outcome  numbers  did  not  seem  to  change.  For  example,  one  professor’s  number 
of  outcomes  did  not  change  from  2005-2006;  however,  the  quality  of  the  outcomes  for 
2006  was  very  significantly  different  and  improved.  Also  that  professor’s  four  primary 
outcomes  were  broken  out  into  second  and  third  level  outcomes.  Again,  for  example,  one 
primary  outcome  inherently  encompassed  five  secondary  outcomes,  with  each  of  those 
broken  out  into  a third  level.  Thus  the  quality  in  content,  linkages,  and  assessments  was 
dramatically  different  and  greatly  improved  for  most  of  the  courses. 
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Table  B.S.b.l:  Standards  ABET-Engineering  Outcomes (Fall  2005  and  Fall  2006  courses) (5  engineering  professors) 


a.  apply 
math, 
science, 
engineering 

b.  design/ 
conduct 
experiments; 
analyze, 
interpret  data 

C. 

design  system, 

component, 

process-given 

constraints, 

etc. 

d. 

function  on 
inter- 
disciplinary 
teams 

e. 

identify,  formulate, 
solve  engineering 
problems 

f. 

understand 

professional, 

ethical 

responsibility 

g- 

ability  to 

communicate 

effectively 

b. 

understand  impact 
eng.  Solutions  on 
global,  economic, 
environment,  society 

recognition  of 
need  for,  and 
ability  to 
engage  in  life- 
long learning 

j- 

knowledge  in 

contemporary 

issues 

k. 

ability  to  use 

techniques, 

skills,  and 

modern 

engineering 

tools 

Fall  2005  and  Fall  200 

6 Courses  - ABET  Outcomes 

5 + 
5+ 

2+ 

Ipartial 

(no 

DOE) 

INR 
1 c 

2+ 

4-1- 

iH-partial 
(could  do 
lots  more) 

1 c 
4+ 

1-1- 

INR 

3c 

2+ 

4-h 

Ic 

5+ 

1-H 

1-1-  (written 
reports  only) 

INR 

2c 

None 

1-H 

1 

3c 

3+ 

1-H 

INR 

2c 

Ic  (minor) 
1+ 

2-1-  1-1- 
(to  small 
effect) 

2c 

2+ 

4-h 

Ic 

4+ 

4- h 

1-H 

(students  don’t 
use  unless 
asked  to) 

5- 1- 

(6-11)  1-4 

1-1 

1-  2 

1-1 

*-8 

1-1 

1 - none 

* - none 

* . 2 

*-3 

(3/6-5)  1-5 

1-1 

1-5 

*-l 

*- 1 

1-1 

(4-4)  1 - 1 

* -none 

1 - none 

* - none 

1-1 

* - none 

*-l 

❖ 

❖ 

1 - 1 

(4-5)  1 - 5 

1-5 

3 

1-5 

* - none 

1-3 

* - none 

* - 1 

* - 1 

1-2 

(4-5)  1-2 

1-1 

1-1 

* - none 

1 - 1 

1 - 1 

Legend:  + = yes-okay;  c = need  to  consider;  other  notes 
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Table  B.5.b.2:  Standards  ABET/TAC/NAIT-Engineering  Technology  & Industrial  Technology 


(2  engineering  technology /tec 

inology  professors) 

(Eall  2005  and  Eall  2006  courses) 

a. 

b. 

c.  ability  to 

d. 

e. 

f. 

g- 

h. 

i. 

j- 

k. 

1. 

m. 

n. 

o. 

P- 

mastery  of 

ability  to 

conduct, 

ability  to 

ability  to 

ability  to 

ability  to 

ability 

recognize 

ability 

respect  for 

commit 

ability  to 

ability 

ability 

ability 

knowledge, 

apply 

analyze, 

apply 

function 

identify, 

commu- 

to  com- 

need  for. 

to 

diversity; 

to  quality, 

program 

to  use 

to 

to 

techniques. 

current 

interpret 

creativity 

effectively 

analyze. 

nicate 

municate 

ability  to 

under- 

know- 

timeliness. 

computers 

modern 

manage 

design, 

skills. 

knowledge; 

experi- 

in  design 

on  teams 

solve 

effectively 

effectively 

engage  in 

stand 

ledge  of 

continuous 

and/or  use 

labor- 

projects 

mani- 

modern  tools 

adapt  to 

ments; 

of  systems, 

technical 

writing 

orally 

lifelong 

profes- 

contempor 

improve- 

computer 

atory 

effect- 

pulate. 

emerging 

apply 

components, 

problems 

learning 

sional. 

ary 

ment 

application 

tech- 

tively 

manage 

applications 

experiment 

processes 

ethical, 

profession- 

s 

niques, 

industri 

of  math. 

al  results  to 

social 

al,  societal, 

effectively 

skills, 

al 

science, 

improve 

responsi 

global 

equip- 

systems 

technology 

processes 

bilities 

issues 

ment 

effect- 

q- 

tively 

ability 

manage 
or  lead 

person- 

nel 

effect- 

tively 

Eall  2005  and  Eall  2006  Courses  - ABET/TAC/NAIT  Outcomes 

2+ 

2+ 

1+ 

1+ 

1+ 

2+ 

1+ 

1+ 

1+ 

1+ 

1-1- 

2+ 

2c 

1+ 

1-1- 

p- 

1 no 

re- 

Ic 

Ic 

Ic 

Ic 

Ic 

Ic 

Ic 

Ic 

Ic 

Ic 

sponse 

Ic 

None 

2+ 

2+ 

2+ 

2+ 

None 

2+ 

2+ 

None 

2+ 

1-1- 

1-1- 

1+ 

1-1- 

1-1- 

None 

9 

not 

q- 

Ino  re- 

sure 

sponse 

Ic 

None 

P- 

(5-6) 

* -NR 

1-6 

1-5 

1- 1 

* - 1 

1-4 

6 

* . 4 

2 

1 

2 

1 

1-1 

NR-NR 

q- 

(6-19) 

*-NR 

1-10 

1-5 

*-5 

* -3 

1-6 

1 

4 

1-one 

1 

*-none 

^"none 

NR-NR 

Legend:  + = yes-okay;  c = need  to  consider;  other  notes 
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Table  B.5.b.3:  NIU  General  Edueation  Goals 


(Fall  2005  and  Fall  2006)  (7  professors  aeross  engineering  and  teehnology) 


Writing 

Speaking 

Listening 

Quantitative 

Reasoning 

Use  of 

Resources- 

Technology 

Historical 
Development 
Of  Culture 

Significance 
of  Arts 

Cultural 

Traditions 

Philosophical 

Ideas 

Methods  in 
Science 
Methods  in 
Social  Science 

Interrelatedness 

Across 

Disciplines 

Social 

Responsihility 

Citizenship 

C C+ 

Earlier  it  was 
only  lab 
reports.  In 
fall  06,  they 
had  to  write 
reports  for 
three  PA 
tasks. 

C C+ 

Presentation 
of  PA  tasks 

+ + 
Listen  to 
guest  speaker, 
professor, 
fellow 
students 
during  PA 
task 

presentations 

+ C+ 

Homework, 

exams,  PA 

tasks  - all 

involved 

quantitative 

reasoning 

+ + 
In  addition 
to  labs  that 
required 
using  many 
resources, 
had  to  use 
outside 
resources 
for  PA 
tasks. 

NR  NA 

NR  NA 

NR  NA 

NR  NA 

NRC 

May  consider 
more  interaction 
with  EE  for 
signal  processing 

NR  C+ 

Discussed  issues 
such  as  energy 
conservation, 
noise,  pollution, 
ethics,  etc. 

C+ 

Did  consider 
and  add,  still 
needs 

improvement; 
will  keep 
working  on  it 

C 

9 

C+ 

Strong,  but 
could  be  better 

+ 

I’m  quite 
pleased 

NA 

NA 

C- 

C+ 

C 

C 

Ok  + 

Project, 

exams, 

homework, 

using  MS 

Word 

C + 

Oral 

presentations 

with 

PowerPoint 

c c- 

Ok  + 

Material 
requires  this 

Ok  + 

Software 

and 

computer  to 

solve 

problems 

NR  NA 

NR  NA 

NR  NA 

C NA 

Ok,  C+ 

Examples, 
exercises  with 
topics  from 
other  disciplines 

C C- 

+ + 
PAs  and 
homework 

+ + 
Pas  and 
discussion 
sessions 

+ + 
Lectures,  case 
studies, 
discussion 
sessions 

+ + 
Problem 
solutions, 
homework, 
midterm  and 
final  exams 

+ + 
PAs  and 
homework 

C NA 

C NA 

C NA 

+ + 
Problem 
solutions, 
homework, 
midterm,  final 
exams,  and  PAs 

+ + 
Case  studies  and 
PAs 

+ C- 

C - + well 

addressed 
through  PA 
reports 

+ NA- 

possible  to 
include  for 
future  course 

C + 

lectures 

+ + 
addressed  in 
project  design 
decisions 

C + 

well 

addressed 

through 

project 

design 

decisions 

NR  NA 

C + 

addressed 
through 
project  design 

NR  - NA 

C + 

well  addressed 
through  project 
design 

C C+ 

to  some  extent 
when  making 
design  decisions 

c c- 

possihle  to 
include  for 
future  courses 

+ + 

Performance 

Tasks 

+ + 
Group 
learning  and 
interactions 

+ + 

Group 

interactions 

+ + 
Performance 
tasks,  lab 
demonstrations 

C NA 

+ C+ 

C + 

Lectures 

+ C+ 

+ NA 

C C+ 

+ C+ 

C + 

C C- 

C C- 

+ N+ 

+ + 

+ NR 

C NA 

C NA 

+ NA 

C NA 

C N\A 

Legend:  + addressed  well;  NA-does  not  really  apply  in  professor’s  opinion;  C-  do  not  do  it,  but  still  need  to  consider  adding  it  in  as  professor  continues  to  make 
changes;  C+  did  consider  and  add  in;  still  needs  improvement  and  professor  will  keep  working  to  improve  or  add; 
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Research  Semester  Results  on  Teaching  Styles  (Fall  2005  and  Fall  2006) 

During  the  initial  course  analysis,  professors  analyzed  their  2005  courses  for  use  of  teaching  styles.  They  referenced  Mosston  and  Ashworth 
(1990)  only.  At  the  end  of  the  research  semester,  professors  were  provided  the  same  Teaching  Styles  list  by  Mosston  and  Ashworth  and  also 
Grasha’s  (1996).  They  were  asked  to  consider  which  styles  were  used  during  the  research  semester.  The  responses  ranged  from  check-offs 
to  comments.  Mosston  and  Ashworth’s  styles  are  compared  for  the  2005  and  2006  courses  on  (Table  5)  below  the  one  for  Grasha  (Table  4). 
Some  professors  estimated  how  many  times  they  used  each  style;  others  made  comments  about  the  ones  they  choose;  and  others  did  both. 
All  professors  made  comparisons  using  Mosston  and  Ashworth’s  teaching  styles,  but  some  professors  also  considered  Grasha’s.  (Mosston 
and  Ashworth  were  provided  during  the  initial  analysis  early  in  the  program.  Later  in  the  program  we  were  trying  to  present  them  with 
varying  options  and  perspectives.  Therefore,  they  were  also  exposed  to  Grasha’s  styles.)  The  most  important  aspect  of  this  reporting  activity 
was  to  reinforce  consideration  of  teaching  styles  and  to  stimulate  a broader  repertoire  of  teaching  styles  or  the  use  of  a greater  variety  of 
teaching  styles  in  their  courses.  Grasha  is  presented  first.  Note:  Outcomes  vary  across  professors,  so  the  two  tables,  Grasha  and 
Mosston  and  Ashworth,  reflect  which  teaching  styles  are  used  across  the  total  of  individual  primary  course  outcomes  by  professor. 
Outcomes  are  presented  by  number  only  in  left  column.  This  program  component  was  very  successful  in  that  professors  varied 
their  teaching  styles  beyond  those  used  in  their  2005  courses. 


Table  B.5.b.4:  Student  Learning  Outcomes  & Teaching  Styles 


(Fall  2006) 


(7  professors  across  engineering  and  technology) 


# of  Outcomes 

Expert 

Formal  Authority 

Personal  Model 

Facilitator 

Delegator 

6 

Yes  -4 

Yes -5 

NR 

Yes -6 

Yes -3 

3-6 

Used  Felder  formally 

NR 

NR 

NR 

NR 

Responded  to  Kolb 

4 

Yes 

Yes 

NR 

NR 

NR 

Used  Kolb  formally 

4 

Yes,  but  used  less  this  time 

Used  for 
fundamentals 

NR 

Used  with  PA  tasks, 
especially  1 & 2 

Used  for  final  PA 
task  #3 

Used  Kolb  formally 

5 

No  Response  to  Grasha 

NR 

NR 

NR 

NR 

Responded  to  Kolb 

6 

No  Response  to  Grasha 

NR 

NR 

NR 

NR 

Responded  to  Kolb 

4 

Yes 

Yes 

No 

Yes 

Yes 

Responded  to  Kolb 
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Table  B.5.2.5:  Student  Learning  Outeomes  & Teaehing  Styles  (Fall  2005  and  Fall  2006) 

(7  professors  across  engineering  and  technology) 


#of 

Outcomes 

Command 

Practice 

Reciprocal 

Self- 

Check 

Inclusion 

Guided 

Discovery 

Convergent 

Discovery 

Divergent 

Production 

Learner 

Designed 

Learner 

Initiated 

Self 

Teaching 

6 

6+ 

yes -10 

6+ 

yes-4 

2c,  4+ 
yes  -5 

6c-l  min. 

6c 

5-I-,  Ic 
yes -20 

6c 

3c  (min) 
3c 

yes  -6 

6c 

6c 

yes  -1 

6c 

3-6 

6+ 

Less  than 
before 

6c 

Much 

more 

5c,  (l  little) 
a few 
times;  did 
not  guide 
oh. 

6c 

more 

than 

normal 

6c 

3-I-,  3c 
more 
than 
before 

1 can  do 
more 

l+,  4c 
more  than 
before 

5c,  1 
little 

6c 

6c 

more 

than 

before 

6c 

more 

than 

before 

4 

2ok 

yes 

2ok,c 

yes 

2c 

yes 

2c 

2c 

NR 

2c 

yes 

NR 

NR 

NR 

NR 

4 

4+ 

used  less 
this  time 

4+ 

2c,  2+ 
used  much 
more 
during 
oral 

discussions 

4c 

4c 

new,  somewhat 
accomplished 
through 
implementation 
of  the  rubrics 

4c 

4c 

used  much 
more  - a 
lot  through 
PA  tasks, 
discussions 

4c 

4c 

4c 

4c 

5 

5+ 

5 some 
used  appx. 

6 times 
mainly/PAs 

NR 

4-1- 

(l  some) 

4c,  INR 

3c,  2NR 

3c,  2NR 

NR 

used  appx. 
12  times  / 
problem 
solving 

NR 

4NR,lc 

NR 

6 

5+,  Ic 
yes  - 20 

5+,  Ic 
yes  - 6 

6c 

yes  - 6 

6c 

6c 

3-I-,  2c 

5c,  1-1- 
yes-3 

5-I-,  Ic 

5c,  1-1- 

6c 

yes -3 

5-I-,  Ic 

4 

4+ 

no;  but,  yes 
with  other 
styles  below 

4+ 

yes,  when 
solving 
problems  in 
class 

4+,  c 

within  the 
group;  but 
without 
professor 
supervision 

4c 

yes; 

feedback 

within 

group 

NR 

yes 

4c-i- 

4c+ 

yes 

4c 

yes, 

sometimes; 
when  the  was 
design 
problems 

4c 

yes 

4c 

yes,  but 
with  some 
guidelines; 
instruction 
is  given 

4c 

yes,  with 
projects; 
but  not  with 
deep 

consultation 
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Research  Semester  Teaching  Models  (Fall  2006) 

In  the  table  below,  the  professors’  analysis  of  their  Fall  2005  course  is  compared  to  what  they 
indicated  actually  occurred  in  the  same,  but  significantly  revised,  course  during  the  research 
semester  of  Fall  2006.  The  Fall  2005  course  is  presented  first  in  black,  and  beneath  that 
information,  the  teaching  models  used  during  the  Fall  2006  course  during  the  research  semester 
are  presented  in  red.  There  are  24  teaching  models;  therefore,  the  complete  list  is  presented  in 
two  charts;  models  are  identified  across  the  first  row.  The  numbers  in  black  represent  what  they 
would  consider  using,  acknowledging  that  in  the  2005  course  those  were  not  in  use.  The 
number  or  comments  in  red  represent  what  they  felt  they  tried  in  the  2006  experimental  course. 

Although  it  may  appear  that  professors  selected  only  a few  new  models  to  use  during  their 
experimental  course  in  2006,  remember  there  are  24  different  models  to  consider.  They  were 
encouraged  to  select  just  a few  models  to  try  out  in  the  2006  courses,  and  then  to  add  other 
models  gradually  in  consecutive  semesters.  Thus,  each  professor  chose  a few  models  to  try  that 
were  different  than  the  most-often  used  “lecture”  model. 

This  aspect  of  the  program  was  successful:  professors  were  exposed  to  24  teaching  models. 
They  used  this  initial  approach  to  analyze  what  models  they  felt  were  used  in  the  2005  course. 
Most  of  them  had  no  previous  knowledge  of  these  models  nor  had  they  considered  “teaching 
models”  at  all,  even  those  who  had  attended  teaching  workshops.  Several  had  been  exposed  to 
“cooperative  learning,”  one  of  the  teaching  models  below,  but  had  not  used  the  model  formally 
at  all  and  only  weakly  structured  informally. 

During  the  teaching  model  program  component,  the  professors  studied  the  24  models  more  in 
depth;  this  was  after  the  initial  analysis  with  a list  and  brief  descriptions.  The  worksheet  used  as 
a study  guide  along  with  the  Joyce,  Weil,  and  Calhoun’s  (2004)  Models  of  Teaching  book 
reveal  how  they  felt  about  each  model  and  whether  they  felt  each  model  had  potential  for  use  in 
teaching  their  course.  That  worksheet  is  presented  later,  as  it  is  a formal  segment  of  the 
program  for  the  redevelopment  of  the  course.  However,  when  reviewing  the  worksheet,  each 
professor’s  comments,  and  then  the  comments  after  the  experimental  course,  one  can  see  the 
growth,  comments,  or  questions. 

After  the  research  semester.  Fall  2006,  we  returned  their  initial  analysis  and  the  study 
worksheet  and  asked  them  to  note  which  teaching  models  they  felt  they  had  actually  used 
during  the  experimental  semester.  Did  they  use  the  ones  that  they  expected  to  try  out?  Did  they 
use  others  not  expected?  The  red  numbers  below  labeled  2006  are  those  responses.  The  data 
reveal  significant  change,  considering  the  context  was  one  course  and  their  first  effort  to 
expand  their  teaching  model  repertoires. 

(See  Teaching  Models  In  Portfolio  Section  B.ll) 
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# Out- 
comes 

Memory 

Progressive 

Part 

Advanced 

Organizers 

Lectnre 

Reciprocal 

Teaching 

Mastery 

Learning 

Cooperative 

Learning 

Graphic 

Organizers 

Concept 

Attainment 

Concept 

Formation 

Concept 

Presen- 

tation 

Conceptnal 

6 

6c 

6c 

5c,2-h 
yes  20-t- 

6-1- 

yes  10 

6c  (1  min.) 

5c,  1-1- 

3-I-,  3c 
yes  3 

3+,  3c  (1  min) 
yes  4 

6c  (1  min) 

6c  (Imin) 

6c 

3-1- ,3c 
yes  6 

3-6 

6c 

6c 

couple  times 
liked  it 

6c 

have  always 
used  it 

6-1- 

do  mnch 

less 

6c 

several 

times 

6c 

nsed,  not 

completely 

rigorons 

6c 

used  much 
more  & 
more  formal 

6c  (1  min) 
nsed  abont  as 
much  as  before 

5c,l-i- 

used...probabl 
y slightly  more 

6c 

6c 

nsed  a 
little 

6c 

used 

4 

NR 

2c 

yes  14 

2c 

yes 

2+ 

yes  14 

2c 

yes  7 

2c 

yes  10 

2ok,c 

yes  7 

2c 

NR 

NR 

2c 

yes 

2c 

4 

NR 

NR 

NR 

yes,  good 
response 

4-1- 

yes,  several 
times 

2c,  2+ 

NR 

2c,  2+ 

yes,  good 
response, 
more 

assessments 
taken  the 
time 

4c 

4c 

4c 

4c 

*Used 
with  *C, 
more 
often 
than 
before; 
now  I 
know 
what  this 
is  called. 

4-1- 

****  nsed 
with  * CP 

5 

2-h 

4-1- 

4-1- 

4+ 

20-used 
frequently 
to  deliver 
course 
materials 

4c 

3-used 
while 
execnting 
PAs; 
demon- 
strated good 
outcomes. 

NR 

4c 

3-  nsed  to 
enhance 
implementat 
ion  of  PAs 

4+ 

NR 

NR 

3c,inr 

4c 

6 

5c 

4-i-,2c 

15 

lectures 

4+,  2c 
3 theory 
linked 
with  lah 
demon- 
stration 

5-I-,  Ic 
20 

lectures 

6c 

6c 

6c 

6-group 

learning 

&PA#3 

5-i-,lc 

10-during 
lectures- 
visual  aids 

6c 

6c 

2c,  4-1- 
6- 

lectures 

on 

funda- 

ment 

4-p,2c 

4 

4c 

used,  but 
not  much 

4-1- 

yes 

4-1- 

yes 

4-1- 

some  parts 
lecture,  but 
not  majority 

4c 

did  this  with 
projects 

4c 

4-1- 

yes,  done 
with  PA 
projects 

4c 

yes,  every 
group  did  that 

4c 

4c 

4-1- 

4-1- 
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# Out- 
comes 

Inductive 

Deductive 

Inquiry 

Simulation 

Jurispruden 

tial 

Direct 

Instruction 

Training 

Synectics 

Psychomotor 

Metaphore 

Non- 

Direct 

Role 

6 

6c 

yes  4 

6c  (1  min) 

6C(2 

min) 

yes  5 

6C(2  min) 

yes  2 

6c 

4-H,2c(1 

min) 

yes  8 

4+,  2c 
yes  5 

6c 

5NR,lc 
yes  2 

NR 
yes  1 

NR 
yes  2 

NR 
yes  2 

3-6 

6c 

used  a lot 

6c 

used  less 
than  before 

2+,  4c 
used  a 
lot 

6c 

extensive 

use 

6c 

6c 

1-I-,  5c 
about  as 
before 

6c 

6c 

some 

6c 

6c 

6c 

4 

NR^ 

NR 

2ok 

NR 

NR 

NR 

NR 

NR 

NR 

NR 

NR 

NR 

NR 

4 

4-t- 

used  much 
more; 
students 
responded 
well!  Used 
past  also. 

4-1- 

used  much 
more; 
students 
responded 
well!  Used 
past  as  well. 

NR 

4c 

NR 

NR 

NR 

done  some 
before;  use- 
ful for  pro- 
blem solving 
procedures; 
excellent  / 
eonceptuali- 
zation;  able 
to  discuss 
different 
approaches 
after  one  as 
presented. 

NR 

NR 

NR 

NR 

NR 

5 

NR 

NR 

4-1- 

7 

4c 

NR 

4c 

10  suitable 
for  certain 
topics 

Ic,  4NR 

NR 

NR 

NR 

3 

4c 

Ic,  3NR 

6 

6c 

9 - Pas  & 
Assignments 

6c 

6c 

4+,  2c 

2 lab  demos 

6c 

2-I-,  4c 

15  lectures 
on  basics 

5c,  1-1- 

2 lab  demos 

6c 

5c,  1-1- 

6c 

6-1- 

5c,  1-1- 

4 

4c 

yes,  hidden 
in  lecture 

4-1- 

yes,  but 
professor 
dos  that 
when 
needed 

4c 

4c 

yes,  students 
simulate 
perfor- 
mance of 
rubrics 

4c 

4c 

4c 

4c 

4c 

4c 

4c 

4c 

(Scarborough,  2006  based  on  Joyce, 


Weil,  & Calhoun,  2004) 
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Student  Learning  Outcomes  & Kolb  (1984)  Learning  Styles 

The  Chart  below  identifies  what  learning  styles,  according  to  Kolb’s  (1984)  styles,  the  professors  felt  they  were  providing  opportunity  for  students  to  use  in  both  the 
2005  and  2006  courses.  It  appears  that  more  attention  was  paid  to  learning  styles  across  professors  in  the  2006  experimental  course.  Two  professors  used  the  Kolb 
Learning  Styles  inventory  formally  with  the  entire  class,  and  a third  professor  used  the  Felder  Learning  Styles  Inventory  formally  with  his/her  class.  This  segment  of  the 
program  was  also  considered  very  successful,  as  it  greatly  enhanced  the  professors’  understanding  of  the  overall  focus  of  teaching  and  the  relationship  between  teaching 
styles,  teaching  models,  and  student  learning  styles.  Their  awareness  was  greatly  increased;  their  understanding  increased;  and,  their  commitment  to  working  on 
increasing  the  diversity  of  teaching  models  and  styles  to  better  engage  a broader  range  of  student  learning  styles  and  to  also  culminate  in  expanding  individual  student 
learning  styles  was  significant.  Below  are  reflections  from  the  three  professors  who  formally  used  LS  Inventories.  (See  Felder  notes) 


Table  B.5.b.7:  Student  Learning  Outcomes  & Kolb  Learning  Styles  2005  & 2006  (7  professors  across  engineering  & technology) 


# Outcomes 

Concrete  Experience 

Abstract  Conceptualization 

Active  Experimentation 

Reflective  Observation 

6 

2c-minimal,  2c,  2+ 

5+,  Ic 

3c,  3+ 

6c 

Also  used  Felder’s 

SL,  IL,  VL,  VL,  AL,  RL, 

GL. 

yes 

yes 

yes 

yes 

3-6 

6+ 

3+,  2c 

4+,  2c 

3+,  1 not  so  much,  2c,  1 a 

Used  Felder’s  only. 

Global  learners  did  better 

Intuitive  learners  did 

Visual  and  verbal  learners  did 

little 

The  notes  are  true  for  the 

than  sequential  learners. 

Better  than  sensing  learners. 

equally  well. 

Reflective  learners  did 

concept  tests,  hut  less  so 
on  the  problem  solving 
tests.  See  write  uu  below. 

better  than  active  learners. 

4 

NR 

2ok,c 

2ok,  c 

2c 

Formally  used  Kolb  with 
students  as  a way  to  show 
students  their  learning  styles. 
Will  use  it  next  time  to  also 
create  activities  tailored  to 
students’  distribution  of  L. 
styles. 

yes 

yes 

yes 

yes 

4 

Formally  used  Kolb’s 

2c,  2+ 

4+ 

4+ 

4+ 

Inventory. 

Yes 

Yes 

yes 

yes 

Concentrated  effort  was 

made  to  have  activities 

to  engage  all  four  (4) 

learning  styles. 

5 

5+ 

INR,  1+,  3c 

INR,  Ic,  3+ 

4+,  Ic 

6 

No  response 

6c 

5+,  Ic 

yes 

4+,  2c 

4 Also  used  Felder’s  SL, 

4+ 

4+ 

5c,  1+ 

2c,  2+ 

VL,  VL,  AL,  RL,  SL,  GL 

yes 

2c,  2+  yes 

Legend:  Black-Kolb  2005  course  analysis;  Red-2006  course  analysis;  Blue-2006  course  using  Felder 
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Reflections  on  using  Felder  & Solomon  Learning  Styles 
B.D.  Collar 

In  the  fall  of 2006,  we  conducted  a research  project  experimentally  investigating  student  learning  in  an  introductory  engineering  mechanics  course.  As 
part  of  the  project,  we  administered  Felder  and  Soloman’s  Index  of  Learning  Styles.  The  survey  is  designed  for  engineering  undergraduates.  It  consists  of 
44  questions  aimed  at  illuminating  students’  preferred  modes  of  learning.  Felder  and  Soloman  characterize  student  learning  styles  with  four  dimensions: 

1.  active  V5.  reflective, 

2.  sensing  V5.  intuitive, 

3.  visual  V5.  verbal, 

4.  sequential  V5.  global. 

In  the  research  project,  we  randomly  split  the  class  into  two  groups.  With  one  group,  we  used  hands-on  manipulatives  to  present  many  of  the  concepts. 
The  second  group  is  a control  group  in  which  we  used  more  traditional  graphical  techniques  to  introduce  and  solidify  concepts. 

A5  it  turned  out,  there  was  no  statistically  significant  difference  in  the  two  groups’  performances  on  objective  performance  tests.  However,  when  we 
examined  the  data  more  closely,  we  did  find  an  interesting  distinction.  Electrical  engineering  students  in  the  experimental  group  did  significantly 
better  than  their  counterparts  in  the  control  group.  It  was  an  effect  not  present  in  the  mechanical  engineering  students,  who  make  up  the  bulk  of  the  class. 
In  fact  mechanical  engineering  students  in  the  control  group  tended  to  do  slightly  better  than  their  counterparts  in  the  experimental  group,  but 
not  by  a statistically  significant  margin. 

It  is  apparent  from  our  data  that  electrical  engineering  students  think  and  learn  differently  than  mechanical  engineering  students.  An  obvious  question  is 
what  makes  the  electrical  engineering  students  more  receptive  to  the  hands-on  teaching  strategy?  When  we  correlated  students’  learning  styles  to  exam 
performance,  we  found  that 

1.  Reflective  learners  tended  to  perform  better  than  active  learners. 

2.  Intuitive  learners  tended  to  perform  better  than  sensing  learners. 

3.  There  was  no  correlation  between  the  visual/verbal  dimension  of  learning  and  exam  performance. 

4.  Global  learners  tended  to  perform  better  than  sequential  learners. 

In  results  I,  2,  and  4 above,  the  p-values  were  all  less  than  0.002.  However,  all  correlation  coefficients  had  magnitudes  on  the  order  of  0.4.  Therefore, 
while  certain  learning  styles  showed  a tendency  for  better  performance,  it  is  clear  that  there  was  no  definite  one-to-one  correspondence.  So  are  the  more 
advantageous  learning  styles  more  prevalent  in  electrical  engineering  students?  The  answer  is  no. 

We  found  no  statistical  difference  between  the  learning  styles  of  electrical  and  mechanical  engineering  students.  In  the  study,  we  tested  for  several  other 
differences  between  mechanical  and  electrical  engineering  students  that  also  correlated  with  exam  scores.  We  were  not  able  to  find  any.  For  now,  the 
difference  is  a mystery. 
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Application  of  Kolb’s  Learning  Styles  to  ISYE370 
By  Reinaldo  Moraga 

I started  my  Operations  Research  class  —ISYE370—  by  giving  the  Kolb’s  learning  inventory  test  to  my  students  in  such  a way  that  they  and  I became  aware  of  the  type  of 
learning  style  they  used  to  learn  and  which  other  styles  they  were  able  to  pursue.  In  addition,  Kolb’s  learning  styles  helped  me  to  improve  the  delivery  of  the  teaching. 

The  test  was  given  to  each  student  in  the  first  class  after  the  presentation  of  the  syllabus.  Then  I explained  to  them  the  importance  of  recognizing  their  preferred  learning 
style  and  how  this  information  could  be  used  for  them  and  me  to  enrich  the  learning  environment  in  the  classroom.  In  addition,  I tried  to  connect  the  importance  of  this 
tool  with  their  professional  career  in  terms  of  communicating  in  the  workforce  and  collaborating  in  groups.  Step  by  step,  I went  through  the  booklet  to  let  them  know  how 

to  fill  the  questionnaire  and  interpret  the  results.  The  students  were  inclined  to  think  that  there  was  a correct  outcome  for  this  test.  Therefore,  I had  to  make  clear  that 
this  was  only  a way  to  diagnose  a r>referred  style  of  learning.  Finally,  I asked  to  take  the  test  home,  answer  the  questionnaire,  and  next  class  give  me  a brief  essay 
reporting  (a)  their  preferred  learning  style,  (b)  actions  they  could  take  to  expand  their  learning  into  other  styles,  and  (c)  which  type  of  activities  in  this  class  could 
produce  connection  with  their  preferred  and  other  learning  styles. 

Most  of  the  students  were  able  to  identify  their  preferred  learning  style.  To  expand  their  learning  styles,  most  of  them  reported  activities  such  as  “exploring  the  world 
around,”  “reading  more  books,"  “doing  more  [hand  work],’’  “being  more  sensitive  to  people’s  feelings,”  “trying  to  make  the  subject  fun  while  learning,”  etc.  Some 
interesting  comments  on  how  to  connect  their  learning  styles  with  my  class  were  “by  becoming  personally  involved  and  influencing  the  others  to  work  together,  ” “to 
have  a review  session  or  a guide  study,  ” and  “to  gather  into  groups  to  think  out  problems.  ” 

I found  this  activity  relevant  because  we  may  use  Kolb’s  test  to  help  us  identify  our  strengths  and  weaknesses  as  instructors,  recognize  our  students'  preferred  styles,  use 
teaching  techniques  to  require  all  learning  styles,  and  encourage  our  students  to  extend  into  other  styles.  Of  particularly  interest  to  me  as  instructor  was  to  learn  the  use 
of  the  learning  cycle  to  design  some  of  my  “lectures.  ” The  learning  cycle  consisted  of  four  questions:  why?,  what?,  how?  and  what  if?  (Harb,  Durrant,  & Terry,  1993.)  I 
tried  to  emphasize  in  my  lectures  the  answers  to  these  questions  because  in  that  way  I could  reach  most  of  the  different  learning  styles  of  my  students.  This  framework 
opened  my  eyes  to  the  importance  of  Kolb’s  learning  styles,  and  because  of  its  practical  applications  in  teaching,  I would  like  to  keep  using  Kolb  learning  cycles  as  part 
of  my  other  classes  I teach  for  the  College  of  Engineering. 

Reference: 

Harb,  Durrant,  & Terry,  (1993).  Use  of  the  Kolb  learning  cycle  and  the  4MAT  system  in  engineering.  Education,  Journal  of  Engineering  Education,  70-77. 
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Use  of  Kolb’s  Inventory  of  Learning  Styles 
CITL  - lENG  475  Fall  2006 
Regina  Rahn 

The  Kolb  Learning  Style  Inventory  was  administered  to  students  in  the  Fall  2006  lENG  475  Decision  Analysis  class.  They  completed  the 
questionnaire  and  interpreted  their  learning  styles.  We  discussed,  as  a class,  the  strengths  of  each  learning  style  and  talked  about  the  types 
of  activities  that  were  useful  for  facilitating  learning  of  each  type.  The  idea  was  to  set  a premise  for  the  assessment  and  instructional 
activities  that  would  be  implemented  during  the  semester. 

In  addition,  we  discussed  ways  that  individuals  could  use  the  knowledge  of  their  learning  styles  to  expand  the  ways  in  which  they  learn  to 
incorporate  other  styles.  The  discussion  included  the  use  of  group  work  (cooperative  learning)  and  peer  review  as  ways  to  aid  in 
accomplishing  this  goal. 

A graduate  student  used  this  as  one  of  the  bases  for  her  graduate  project.  The  project  was  completed  at  the  end  of  the  semester.  The  lENG 
475  students  were  surveyed  about  their  thoughts  in  regard  to  the  use  of  the  learning  style  inventory.  The  responses  were  extremely  positive, 
and  they  definitely  saw  the  value  in  the  exercise. 

The  Decision  Analysis  class  also  posed  a unique  opportunity  for  discussions  surrounding  learning  styles.  We  investigated  relationships 
between  learning  styles  and  peoples  ’ attitude  toward  risk,  which  is  a key  element  in  the  course  subject  matter.  I intend  to  continue  utilizing 
learning  style  inventories  in  my  courses. 


Student  Learning  Outcomes  & Bloom  & Dale  (Fall  2005  and  Fall  2006) 

The  professors  analyzed  their  student  learning  outcomes  against  Bloom’s  Learning  Dimensions  and  Dale’s  Cone  of  Learning.  The  analysis  of  the  2005  course  is 
presented  in  black  below  and  also  as  a composite,  number  of  outcomes  achieving  what  level  on  Bloom’s  and  Dale’s  models.  The  2006  course  analysis,  however, 
is  presented  in  red.  Dale’s  levels  are  presented  by  number  of  outcomes  and  level  of  the  Cone.  For  Bloom,  each  outcome  is  listed  at  the  level  achieved.  This 
program  component  was  also  successful.  The  professors  really  seemed  to  grasp  Bloom’s  intentions,  whether  traditional  or  revised.  They  not  only  benefited  from 
using  it  as  an  analysis  tool,  but  in  the  later  re-development  of  their  courses.  They  also  grasped  Dale’s  intentions  about  passive  versus  active  learning.  These 
models  seemed  to  build  good  initial  awareness,  which  deepened  as  they  used  them  as  tools  more  and  more,  beginning  with  the  initial  2005  analysis  and  then  as  a 
metric  for  the  re-development  of  the  2006  courses.  There  was  significant  change  in  the  quality  of  their  student  learning  outcomes.  The  professors’  student 
learning  outcomes  were  developed  and  written  to  achieve  higher  cognitive  processing  levels  on  Bloom’s  Cognitive  Dimension.  T he  outcomes  also  reflective 
higher  quality  in  that  they  reflected  more  active  learning.  The  outcomes  reflected  a potentially  higher  level  of  critical  thinking  as  well.  This  program  component 
resulted  in  significant  change  and  left  them  with  simple  tools  to  use  as  metrics  for  ongoing  change  and  quality  checks. 
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Table  B.5.b.8:  Student  Learning  Outcomes  & Bloom  & Dale  (Fall  2005  and  Fall  2006) 
(7  professors  across  engineering  and  technology) 


# Outcomes: 

Dale's  Cone 

Knowledge 

Comprehension 

Application 

Analysis 

Synthesis 

Evaluate 

Critical 

05  reported  as  composite 

Levels  : P 

Remember 

Understand 

Apply 

Analyze 

Evaluate 

Create 

Thinking  Level: 

06  reported-  specific  outcome 

AA+ 

LM  H 

(1)  6 outcomes 

NR 

1+,  5NR 

2-I-,  4NR 

2+,  4NR 

Ic,  5NR 

1+,  5NR 

6NR 

2Lm  3M,  IH 

composite 

1-11 

1,  3,  7.  8.  9 

2,  5,  6,  7. 10, 11 

1,4 

1-11  numbers  1-11 

(8-10)  A-i- 

3P,  lA 

6-1- 

6-1- 

3-I-,  3c 

5c,  INR 

6c 

6c 

2L,  2L+,  IL/M,  IM; 

(2)  3/6 

2A- 

1-3/6 

2Mc 

1-5 

5 

(10)  At- 

2PA-C 

2-i-c 

2-i-c 

2-I-C 

2-I-C 

2-I-C 

2c 

2Mc 

(3)  2 

1-2 

1,2 

2 

(8-10)  At- 

2A,  2A-I- 

4-1- 

4-1- 

4-1- 

4-1- 

Ic,  l-i--,  2-1- 

4c 

2M,  2H 

(4)  4 

NR 

NR 

NR 

NR 

NR 

NR 

NR 

2P,  2A,  1A+ 

2+,  3NR 

2NR,  3+ 

2NR,  3+ 

NR 

NR 

NR 

3M 

(5)  5 

1(6); 

19 

2-3  (9) 
4(6) 

5-19 

(9- 10)  At- 

NR 

NR 

NR 

NR 

NR 

NR 

4P,2A 

3(2) 

2-I-,  4c 

3-I-,  3c 

5-I-,  Ic 

4-I-,  2c 

5-I-,  Ic 

4c,  2-1- 

3L,  2M,  IH 

(6)  6 

4 (1,3,5) 

1,  2,  3,  5 

4 

6 

6 

8-9  (1) 
8(1) 

IP,  IP-A, 

4-1- 

4-1- 

1-i-c,  3-1- 

Ic,  let-,  2-1- 

2c,  2-1- 

2c,  2-1- 

2c,  2c-i- 

2A-P 

1 

2 

3,4 

(7)  4 

4 

1-4  (1-10) 

Bloom  (1956);  Anderson  & Krathwohl  (2001);  Legend  for  Blooms  levels:  NR  = no  response;  number  + = number  of  outcomes  at  that  level;  +c  = okay,  but  still  need  to  consider;  c = 
need  to  consider  achieving;  c+  = some  positive  accomplishment,  hut  still  needs  work  (e.g.,  outcome  number  reported  by  each  Boom  level) 

Dale  (1969):  Legend  for  Dale’s  levels:  9-10  = active  learning-doing  level;  8 = active  learning-participating;  3-7  = visual  receiving/passive;  2-1  = verbal  receiving-passive, 

(e.g.,  outcome  number  - level) 
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Table  BS.b.l.l:  Faculty  Sample:  GAPS  Analysis  Summary-Abdel  Motaleb 


Standards  ABET-Engineering 


a.  apply 
math, 
science, 
engineering 

b.  design/conduct 
experiments; 
analyze,  interpret 
data 

c.  design  system,  component, 
process-given  constraints,  etc. 

d.  function  on 

interdisciplinary 

teams 

e.  identify,  formulate, 
solve  engineering 
problems 

f.  understand 
professional, 
ethical 

responsibility 

g.  ability  to 

communicate 

effectively 

h.  undst. 
impact  eng. 
Sol  global, 
economic, 
evnir., 
society 

i.  recognition  of 
need  for,  and 
ability  to  engage  in 
life-long  learning 

j.  Knowledge  in 

contemporary 

issues 

k.  ability  to  use  techniques,  skills, 
and  modern  engineering  tools 

V 

C 

V 

C 

V 

C 

C 

C 

C 

V 

V 

NIU  General  Education 


Writing 

Speaking 

Listening 

Quantitative 

Reasoning 

Use  of  Resources-Technology 

Historical 
Development 
Of  Culture 

Significance  of 
Arts 

Cultural  Traditions 
Philosophical  Ideas 

Methods  in  Science 
Methods  in  Social  Science 

Interrelatedness 
Across  Disciplines 

Social  Responsibility 
Citizenship 

c 

c 

c 

V 

V 

V 

c 

c 

V 

c 

c 

Student  Learning  Objectives/Outcomes  & Teaching 


Models 


Objectives 

Mem 

Prog 

Part 

Adv 

O 

Lee 

Rec 

Tch 

Mast 

Learn 

Coop 

Learn 

Graphic 

Org 

Concept 

Attainm 

Cone 

Form 

Cone 

Pres 

Con- 

ceptual 

Induct 

Deduct 

Inquiry 

Sim- 

ulate 

JurisP 

Direct 

Instr 

Train 

Synect 

Psycho- 

motor 

Meta- 

phore 

Non- 

direct 

Role 

Give  students 
an 

introduetion 

to 

semieonduetor 

material 

properties 

c 

c 

V 

V 

c 

c 

V 

c 

c 

c 

V 

V 

c 

V 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

Learn  the 
basie  theories 
of  modem 
electronic 
devices 

c 

c 

V 

V 

c 

c 

V 

c 

c 

c 

V 

V 

c 

V 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 
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Student  Learning  Objectives/Outcomes  & Teaching  Styles 


Objectives 

Command 

Practice 

Reciprocal 

Self-Check 

Inclusion 

Guided 

Discovery 

Convergent 

Discovery 

Divergent 

Production 

Learner 

Designed 

Learner 

Initiated 

Self 

Teaching 

Give  students  an  introduction  to  semiconductor 
material  properties 

V 

V 

V,c 

c,V 

c,V 

c,V 

c 

c 

c 

c 

Learn  the  basic  theories  of  modem  electronic 
devices 

V 

V 

V,c 

c,V 

c,V 

c,V 

c 

c 

c 

c 

Apply  semiconductor  theories  to  design 
electronic  devices  and  investigate  their 
performance. 

V 

V 

V,c 

c,V 

c,V 

c,V 

c 

c 

c 

c 

Conduct  a design  project  using  MathCAD  to 
design  different  types  of  devices. 

V 

V 

V,c 

c,V 

c,V 

c,V 

c 

c 

c 

c 

Notes:  First  symbol  means  the  majority  of  activity  and  the  second  is  the  minority.  V,  C means  this  is  mostly  done  but  needs  to  be  refined  or  activity  increased.  C,V  means  this  is 
not  generally  done,  but  sometimes  it  is. 
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Student  Learning  Objectives/Outcomes  & Learning  Styles 


Objectives 

Concrete  Experience 

Abstract  Conceptualization 

Active  Experimentation 

Reflective  Observation 

Give  students  an  introduction  to 
semiconductor  material 
properties 

V 

V 

C 

C 

Learn  the  basic  theories  of 
modem  electronic  devices 

V 

V 

C 

C 

Apply  semiconductor  theories 
to  design  electronic  devices  and 
investigate  their  performance 

V 

V 

V 

V 

Conduct  a design  project  using 
MathCAD  to  design  different 
types  of  devices 

V 

V 

V 

V 
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Student  Learning  Objectives/Outcomes  & Bloom  & Dale 


Objectives 

Dale's  Cone 
Levels  P A 
A+ 

Knowledge 

Remember 

Comprehension 

Understand 

Application 

Apply 

Analysis 

Analyze 

Synthesis 

Evaluate 

Create 

Critical 
Thinking 
Level  LM  H 

Give  students  an  introduction  to 
semiconductor  material 
properties 

P 

V 

V 

V,c 

C 

C 

C 

C 

Learn  the  basic  theories  of 
modem  electronic  devices 

P,A 

V 

V 

V 

c,V 

C 

C 

C 

Apply  semiconductor  theories 
to  design  electronic  devices  and 
investigate  their  performance 

A,P 

V 

V 

V 

V 

V 

V 

c,V 

Conduct  a design  project  using 
MathCAD  to  design  different 
types  of  devices 

A,P 

V 

V 

V 

V 

V 

V 

c,V 
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Table  B.5.b.2.1:  GAPS  Analysis  Summary  - lENG  475,  Decision  Analysis:  Regina  Rahn 


Standards  ABET-Engineering 


a.  apply 
math, 
science, 
engineering 

b.  design/conduct 
experiments; 
analyze,  interpret 
data 

c.  design  system,  component, 
process-given  constraints,  etc. 

d.  function  on 

interdisciplinary 

teams 

e.  identify,  formulate, 
solve  engineering 
problems 

f.  understand 
professional, 
ethical 

responsibility 

g.  ability  to 

communicate 

effectively 

h.  undst. 
impact  eng. 
Sol  global, 
economic, 
evnir., 
society 

i.  recognition  of 
need  for,  and 
ability  to  engage  in 
life-long  learning 

j.  Knowledge  in 

contemporary 

issues 

k.  ability  to  use  techniques,  skills, 
and  modern  engineering  tools 

v+ 

V- 

V-H 

C 

V-H 

V 

V 

V- 

V- 

V- 

V 

Standards  ABET/' 


'AC/NAIT-Engineering  Technology  and  Industrial  Technology 


a.  mastery 
of 

knowledge, 

techniques, 

skills, 

modern 

tools 

b.  ability  to 

apply 

current 

knowledge; 

adapt  to 

emerging 

applications 

of  math, 

science, 

technology 

c.  ability  to 

conduct, 

analyze, 

interpret 

experiments; 

apply 

experimental 
results  to 
improve 
processes 

d.  ability  to 
apply 

creativity  in 
design  of 
systems, 
components, 
processes 

e.  ability 
to 

function 
effectively 
on  teams 

f.  ability 
to 

identify, 

analyze, 

solve 

technical 

problems 

g.  ability  to 
communicate 
effectively 
writing 

h.  ability  to 
communicate 
effectively 
orally 

recognize 
need  for, 
ability  to 
engage  in 
lifelong 
learning 

j.  ability  to 
understand 
professional, 
ethical,  social 
responsibilities 

k.  respect  for 

diversity; 

knowledge  of 

contemporary 

professional, 

societal, 

global  issues 

1.  commit 
to  quality, 
timeliness, 
continuous 
improvement 

m.  ability  to 
program 
computers 
and/or  use 
computer 
applications 
effectively 

n.  ability  to 

use  modern 

laboratory 

techniques, 

skills, 

equipment 

effectively 

o.  ability  to 
manage 
projects 
effectively 

p.  ability  to 

design, 

manipulate, 

manage 

industrial 

systems 

q.  ability 
to  manage 
or  lead 
personnel 
effectively 

NIU  General  Education 


Writing 

Speaking 

Listening 

Quantitative 

Reasoning 

Use  of  Resources-Technology 

Historical 
Development 
Of  Culture 

Significance  of 
Arts 

Cultural  Traditions 
Philosophical  Ideas 

Methods  in  Science 
Methods  in  Social  Science 

Interrelatedness 
Across  Disciplines 

Social  Responsibility 
Citizenship 

V 

V 

V 

V 

V 

c 

c 

c 

V 

V 

c 
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Student  Learning  Object 

tives/Outcomes  & Teachin 

g Models 

Objectives 

Mem 

Prog 

Part 

Adv 

O 

Lee 

Ree 

Teh 

Mast 

Learn 

Coop 

Learn 

Graphic 

Org 

Concept 

Attainm 

Cone 

Form 

Cone 

Pres 

Con- 

ceptual 

Induct 

Deduct 

Inquiry 

Sim- 

ulate 

JurisP 

Direct 

Instr 

Train 

Synect 

Psycho- 

motor 

Meta- 

phore 

Non- 

direct 

Role 

To  study  the 
meaning  and 
application  of 
analytic 
decision 
making  ... 

V 

c 

c 

c 

c 

c 

c 

V 

V 

V 

c 

To  learn  a 
specific  set  of 
analytic  tools. 
These  tools 
are 

applicable  to 
technical 
decision 
which  ... 

V 

c 

c 

c 

c 

c 

c 

V 

V 

V 

c 

To  become 
aware  of  the 
limitations  of 
the  models 
for  rational 
decision 
making. 

V 

V 

V 

c 

c 

c 

c 

V 

V 

V 

c 

To  allow 
students  to 
practice  the 
structuring 
and  solution 
of 

complicated 

decision 

problems. 

V 

V 

c 

c 

c 

c 

c 

V 

V 

V 

c 
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Student  Learning  Objectives/Outcomes  & Teaching  Styles 


Objectives 

Command 

Practice 

Reciprocal 

Self-Check 

Inclusion 

Guided 

Discovery 

Convergent 

Discovery 

Divergent 

Production 

Learner 

Designed 

Learner 

Initiated 

Self 

Teaching 

To  study  the  meaning  and  application  of  analytic  decision 
making  techniques  for  technical  decision  making  under 
uncertainty. 

V 

V 

c 

c 

c 

c 

c 

c 

c 

c 

c 

To  learn  a specific  set  of  analytic  tools.  These  tools  are 
applicable  to  technical  decisions  that  must  he  made  when 
present  or  future  states  of  nature  are  uncertain,  and  multiple 
attributes  are  considered.  Included  in  these  methods  are 
decision  tree  methodology  and  utility  theory. 

V 

V 

c 

c 

c 

c 

c 

c 

c 

c 

c 

To  become  aware  of  the  limitations  of  the  models  for  rational 
decision  making. 

V 

V 

V 

c 

c 

c 

c 

c 

c 

c 

c 

To  allow  students  to  practice  the  structuring  and  solution  of 
complicated  decision  problems. 

V 

V 

V 

c 

c 

c 

c 

c 

c 

c 

c 
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Student  Learning  Objectives/Outcomes  & Learning  Styles 


Objectives 

Concrete  Experience 

Abstract  Conceptualization 

Active  Experimentation 

Reflective  Observation 

To  study  the  meaning  and  application 
of  analytic  decision  making  techniques 
for  technical  decision  making  under 
uncertainty. 

C 

V 

V 

V 

To  learn  a specific  set  of  analytic  tools. 
These  tools  are  applicable  to  technical 
decisions  that  must  he  made  when 
present  or  future  states  of  nature  are 
uncertain,  and  multiple  attributes  are 
considered.  Included  in  these  methods 
are  decision  tree  methodology  and 
utility  theory. 

C 

V 

V 

V 

To  become  aware  of  the  limitations  of 
the  models  for  rational  decision 
making. 

V 

V 

V 

V 

To  allow  students  to  practice  the 
structuring  and  solution  of  complicated 
decision  problems. 

V 

V 

V 

V 
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Student  Learning  Objectives/Outcomes  & Bloom  & Dale 


Objectives 

Dale's  Cone 
Levels  P A 
A+ 

Knowledge 

Remember 

Comprehension 

Understand 

Application 

Apply 

Analysis 

Analyze 

Synthesis 

Evaluate 

Create 

Critical 
Thinking 
Level  LM  H 

To  study  the  meaning  and  application  of 
analytic  decision  making  techniques  for 
technical  decision  making  under 
uncertainty. 

A 

V 

V 

V 

V 

C 

C 

M 

To  learn  a specific  set  of  analytic  tools. 
These  tools  are  applicable  to  technical 
decisions  that  must  he  made  when 
present  or  future  states  of  nature  are 
uncertain,  and  multiple  attributes  are 
considered.  Included  in  these  methods 
are  decision  tree  methodology  and 
utility  theory. 

A 

V 

V 

V 

V 

V- 

C 

M 

To  become  aware  of  the  limitations  of 
the  models  for  rational  decision  making. 

A+ 

V 

V 

V 

V 

V 

C 

H 

To  allow  students  to  practice  the 
structuring  and  solution  of  complicated 
decision  problems. 

A+ 

V 

V 

V 

V 

V 

C 

H 
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Table  B.5.b.3.1:  GAPS  Analysis  Summary 

(+  means  OK,  C means  may  be  Considered  in  future) 

Standards  ABET-Engineering 


a.  apply 
math, 
science, 
engineering 

b.  design/conduct 
experiments; 
analyze,  interpret 
data 

c.  design  system,  component, 
process-given  constraints,  etc. 

d.  function  on 

interdisciplinary 

teams 

e.  identify,  formulate, 
solve  engineering 
problems 

f.  understand 
professional, 
ethical 

responsibility 

g.  ability  to 

communicate 

effectively 

h.  undst. 
impact  eng. 
Sol  global, 
economic, 
evnir., 
society 

i.  recognition  of 
need  for,  and 
ability  to  engage  in 
life-long  learning 

j.  Knowledge  in 

contemporary 

issues 

k.  ability  to  use  techniques,  skills, 
and  modern  engineering  tools 

+ 

+ (not  much  design 
of  expt) 

C (only  to  small  extent  is 
done  now) 

C 

+ 

C 

+ (so  far  only 

written 

reports) 

c 

C (only  to  small 
extent  is  done) 

+ 

+ 

Standards  ABET/' 


AC/NAIT-Engineering  Technology  and  Industrial  Technology 


a.  mastery 
of 

knowledge, 

techniques, 

skills, 

modern 

tools 

b.  ability  to 

apply 

current 

knowledge; 

adapt  to 

emerging 

applications 

of  math, 

science, 

technology 

c.  ability  to 

conduct, 

analyze, 

interpret 

experiments; 

apply 

experimental 
results  to 
improve 
processes 

d.  ability  to 
apply 

creativity  in 
design  of 
systems, 
components, 
processes 

e.  ability 
to 

function 
effectively 
on  teams 

f.  ability 
to 

identify, 

analyze, 

solve 

technical 

problems 

g.  ability  to 
communicate 
effectively 
writing 

h.  ability  to 
communicate 
effectively 
orally 

recognize 
need  for, 
ability  to 
engage  in 
lifelong 
learning 

j.  ability  to 
understand 
professional, 
ethical,  social 
responsibilities 

k.  respect  for 

diversity; 

knowledge  of 

contemporary 

professional, 

societal, 

global  issues 

1.  commit 
to  quality, 
timeliness, 
continuous 
improvement 

m.  ability  to 
program 
computers 
and/or  use 
computer 
applications 
effectively 

n.  ability  to 

use  modern 

laboratory 

techniques, 

skills, 

equipment 

effectively 

0.  ability  to 
manage 
projects 
effectively 

p.  ability  to 

design, 

manipulate, 

manage 

industrial 

systems 

q.  ability 
to  manage 
or  lead 
personnel 
effectively 

NIU  General  Education 


Writing 

Speaking 

Listening 

Quantitative 

Reasoning 

Use  of  Resources-Technology 

Historical 
Development 
Of  Culture 

Significance  of 
Arts 

Cultural  Traditions 
Philosophical  Ideas 

Methods  in  Science 
Methods  in  Social  Science 

Interrelatedness 
Across  Disciplines 

Social  Responsibility 
Citizenship 

Only 
lab 
rep.  c 

C 

C 

+ 

+ 
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Student  Learning  C 

►bjeci 

;ives/( 

Outcomes  & Teaching  ] 

Models 

Objectives 

Me 

111 

Pr 

og 

Pa 

rt 

Ad 

V 

O 

Le 

c 

Re 

c 

Tc 

h 

Mas 

t 

Lear 

n 

Coop 

Lear 

n 

Graph 

ic 

Org 

Conce 

pt 

Attain 

m 

Cone 

For 

m 

Cone 

Pres 

Con- 

ceptua 

1 

Induct 

Deduct 

Inquiry 

Sim- 

ulate 

JurisP 

Dire 

ct 

Instr 

Trai 

n 

Synect 

Psycho- 

motor 

To  learn  basic 

theories 

behind 

vibrating  mass 
or  equipment 

c 

c 

+ 

+ 

c 

+ 

c 

+ 

c 

c 

c 

+ 

c 

c 

minim 

ally 

c 

c 

c 

-1- 

-1- 

c 

To  familiarize 
with  various 
equipment  and 
software  used 
to  run 
experiment 

c 

c 

+ 

+ 

c 

c 

+ 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

-1- 

-1- 

c 

To  perform 
experiment  to 
verify  theories 

c 

c 

+ 

+ 

c 

C 

(min 

imal 

ly) 

c 

C 

(mini 

mally) 

c 

C 

(mini 

mall 

y) 

c 

c 

c 

c 

c 

C 

(mini 

mall 

y) 

c 

-1- 

-1- 

c 

c 

To  learn  how 
to  write  a 
laboratory 
report 

c 

c 

c 

+ 

c 

c 

+ 

+ 

c 

c 

c 

c 

c 

c 

C 

(minimal 

ly) 

c 

c 

-1- 

-1- 

c 

To  apply 
major 

commercially 

available 

mathematical 

and 

engineering 

software 

c 

c 

+ 

+ 

C 

(m 

ini 

m 

all 

y 

c 

+ 

+ 

c 

c 

c 

+ 

c 

c 

c 

(minimal 

ly) 

C 

(mini 

mall 

y) 

c 

C(mi 

nima 

iiy) 

c 

c 

To  design 
structures 
where  failure 
from  vibration 
is  prevented 

c 

c 

+ 

+ 

c 

c 

c 

c 

C 

(mini 

mally) 

c 

c 

+ 

c 

c 

+ 

c 

c 

c 

c 

c 
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Student  Learning  Objectives/Outcomes  & Teaching  Styles 

Objectives  Command  Practice  Reciprocal  Self-Check  Inclusion  Guided  Convergent  Divergent  Learner  Learner  Self 


Discovery 

Discovery 

Production 

Designed 

Initiated 

Teaching 

To  learn  basic  theories  behind  vibrating  mass 
or  equipment 

+ 

+ 

c 

C(only 

minimally) 

c 

+ 

C 

c 

(minimally) 

C 

C 

C 

To  familiarize  with  various  equipment  and 
software  used  to  run  experiment 

-1- 

-1- 

-1- 

c 

c 

-1- 

C 

C (minimally) 

C 

C 

C 

To  perform  experiment  to  verify  theories 

-1- 

-1- 

-1- 

c 

c 

-1- 

C 

c 

C 

C 

C 

To  learn  how  to  write  a laboratory  report 

-1- 

-1- 

-1- 

c 

c 

c 

C 

c 

C 

C 

c 

To  apply  major  commercially  available 
mathematical  and  engineering  software 

-1- 

-1- 

-1- 

c 

c 

+ 

C 

c 

c 

C 

c 

To  design  structures  where  failure  from 
vibration  is  prevented 

-1- 

-1- 

c 

c 

c 

-1- 

C 

C 

(minimally) 

c 

C 

c 
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Student  Learning  Objectives/Outcomes  & Bloom  & Dale 


Objectives 

Dale's  Cone 
Levels  P A 
A+ 

Knowledge 

Remember 

Comprehension 

Understand 

Application 

Apply 

Analysis 

Analyze 

Synthesis 

Evaluate 

Create 

Critical 
Thinking 
Level  LM  H 

To  learn  basic  theories  behind 
vibrating  mass  or  equipment 

+ 

C 

(minimally) 

M 

To  familiarize  with  various 
equipment  and  software  used  to 
run  experiment 

+ 

L 

To  perform  experiment  to  verify 
theories 

+ 

M 

To  learn  how  to  write  a 
laboratory  report 

+ 

L 

To  apply  major  commercially 
available  mathematical  and 
engineering  software 

+ 

M 

To  design  structures  where 
failure  from  vibration  is 
prevented 

+ 

H 
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DISCUSSION  OF  PROFESSORS’  2005  AND  2006  TEST  AND  TEST  ANALYSIS 

COMPARISONS 
Jerry  Gilmer,  Ph.D. 

Each  professor  submitted  eopies  of  the  midterm  and  final  exams  they  used  in  their  classes 
during  the  Fall  2005  semester  and  the  corresponding  exams  they  used  during  the  Fall  2006 
semester.  The  Fall  2006  exams  were  developed  during  and  after  formal  training  in  test 
development;  it  is  expeeted  that  the  2006  exams  would  be  improved  over  the  2005  exams. 
The  professors  also  learned  formal  item  analysis  proeedures  and  generated  item  analyses  for 
their  exams  from  both  Fall  2005  and  Fall  2006  semesters.  The  Test  Analysis  and 
Development  eomponent  of  the  CEET  Faculty  Development  Program  served  as  a variable  to 
determine  program  sueeess.  Thus,  the  capability  the  professors  developed  to  analyze  tests 
and  the  overall  quality  improvement  of  the  new  tests  for  the  2006  course  were  the  factors 
used  in  the  researeh  and  evaluation  design  for  the  program. 

The  professors’  exams  and  the  item  analysis  results  were  eompared  and  some  general 
comments  from  the  comparisons  are  included  below.  During  the  test  development  training 
professors  learned  about  different  characteristics  of  good  exams,  and  it  is  these 
eharaeteristies  that  were  used  for  making  comparisons  from  2005  to  2006.  Characteristics  of 
item  analysis  were  also  presented  to  the  professors  and  considered  for  comparisons. 

The  diseussion  below  ineludes  a brief  statement  regarding  the  meaning  of  each  characteristic 
and  then  a brief  evaluation  related  to  the  changes  in  tests  and  item  analyses  from  2005  to 
2006.  The  eharaeteristies  were  applied  for  evaluation  to  eaeh  of  the  exams  and  analyses 
submitted  by  eaeh  professor. 

Test  Characteristics 


Based  on  Item  Bank  - Is  the  exam  based  on  an  item  bank  - a pool  of  items?  During  the  test 
development  sessions,  professors  were  asked  to  develop  an  item  bank  covering  all  of  the 
content  areas  in  their  eourse. 

Evaluation  - It  appears  that  none  of  the  exams  for  2005  were  based  on  an  item  bank  and 
that  all  of  the  exams  for  2006  were  based  on  an  item  bank. 

Based  on  Table  of  Specifications  - Is  the  exam  based  on  a formal  plan,  a blueprint,  also 
called  a table  of  specifications?  The  professors  were  taught  how  to  create  and  utilize  a table 
of  specifieations  to  ensure  the  test  eovers  all  the  intended  material  in  appropriate/specified 
proportions. 

Evaluation  - It  appears  that  none  of  the  exams  for  2005  were  based  on  a table  of 
speeifications  and  that  all  of  the  exams  for  2006  were  based  on  a table  of 
specifications. 

Overall  Appearance  - well  laid  out;  pleasing  appearance,  grammar,  etc.  - The  exam 
should  be  generally  pleasing  to  look  at  and  easy  to  read  and  follow,  eharaeteristies  sueh  as 
font  selection,  spacing,  use  of  highlights  such  as  indents,  bold  letters,  etc.  should  be 
considered. 
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Evaluation  - The  2006  exams  were  much  improved  over  the  2005  exams  for  six  of  the 
seven  professors.  In  general  appearance,  the  2006  exams  from  one  professor  looked 
similar  to  the  2005  exams. 

Overall  Instructions  - clear,  unambiguous  - The  exam  should  include  some  overall 
instructions  to  the  students  and  these  instructions  should  appear  on  the  first  page  of  the  exam. 

Evaluation  - Four  professors  included  very  good  and  improved  instructions  for  their 
2006  exams.  Two  instructors  did  not  provide  any  overall  instructions  and  one 
professor  modified  the  instructions  only  slightly. 

Instructions  for  Item  Subsets  - clear  unambiguous  - Sometimes  instructions  are 
necessary  for  item  sets  - a group  of  multiple  choice  items  or  a group  of  short-answer  items, 
for  example,  or  a set  of  items  related  to  a common  diagram  or  a common  reading  passage. 

Evaluation  - Four  of  the  professors  included  item  sets  and  their  instructions  for  these 
sets  for  2006  were  generally  very  good.  Three  of  the  professors  did  not  use  item  sets. 

Number  of  short,  discrete  items  vs.  longer  items  - The  test  development  sessions  covered 
the  use  of  more  short,  discrete  items  rather  than  fewer  longer  items  to  ensure  that  the  exam 
provides  an  adequate  sampling  of  the  course  material. 

Evaluation  - All  of  the  2006  exams  were  improved  on  this  characteristic.  Most 

professors  used  long,  problem-type  items  for  their  2005  exams  but  converted  to  more 
discrete  items  for  their  2006  exams.  Two  professors  added  only  a small  number  of 
discrete  items  and  still  used  few,  problem- type  items. 

Number  of  objectively  scored  vs.  subjectively  scored  items  - Although  subjectively  scored 
items  (short-  or  long-answer,  problem-type,  requiring  scorer  judgments)  are  sometimes 
appropriate,  the  use  of  objectively  scored  items  (multiple  choice,  true-false,  etc.)  is  often 
preferred.  They  are  easier  and  more  efficient  to  score,  and  generally  yield  a more  reliable 
exam. 

Evaluation  - All  professors  (along  with  using  more  short,  discrete,  items)  used  items  that 
could  be  objectively  scored.  Most  professors  included  a few  subjectively  scored 
items  as  well.  Two  professors  used  only  subjectively  scored  items  on  the  final  exam; 
one  of  those  did  so  after  discussing  the  midterm  with  the  class  and  determining  that 
subjectively  scored  items  were  more  appropriate  to  the  course  content. 

Item  Quality  - clear,  direct,  well  written,  no  clues  - The  items  should  be  well- written, 
using  appropriate  English,  and  should  contain  no  clues  or  cues. 

Evaluation  - This  characteristic  is  difficult  for  a non-expert  in  the  content  to  judge,  but 
generally  the  exams  appeared  to  be  clear  and  understandable.  For  two  professors,  the 
2006  exams  seemed  to  exhibit  no  changes.  For  the  other  five  professors,  the  2006 
exams  exhibited  noticeable  improvement.  But  the  2005  exams  appeared  to  be 
satisfactory  to  begin  with. 
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Item  Analysis  Characteristics 

Item  Difficulty  - number  of  very  difficult  items  - In  general,  the  items  should  not  be  too 

difficult.  If  less  that  50%  of  possible  points  for  an  item  were  awarded,  the  item  was  judged  to 
be  too  difficult.  This  threshold  (50%)  is  not  inappropriate  and  was  used  consistently  to 
evaluate  the  exams  and  item  analyses  submitted  by  the  professors.  But  the  professors  were 
informed  that  they  should  determine  their  own  threshold,  one  that  is  meaningful  to  them  and 
their  exams  - that  there  is  no  specific  value  that  should  be  used  for  judging  items  in  all 
situations. 

Evaluation  - Some  professors’  2005  exams  contained  a small  number  of  problems,  and 
these  problems  had  generally  high  (easy)  difficulty  indices.  All  the  analyses  for  the 
2006  exam  analyzed  several  more  items.  It  appeared  that  more  items  were  flagged  as 
being  too  difficult  (difficulty  index  less  than  50%)  than  would  normally  be  desired. 
New  item  types  were  being  used  for  the  first  time  by  most  of  the  professors  and  they 
are  still  learning  how  to  use  item  analysis  data  to  improve  item  performance  for  future 
administrations. 

Item  Discrimination  - number  of  poorly  discriminating  items  - In  general,  the  items 
should  discriminate  between  students  proficient  in  the  content  and  students  who  are  less 
proficient.  Items  were  flagged  if  the  discrimination  index  was  less  than  0.15.  This  threshold 
in  not  completely  arbitrary;  however,  as  with  the  difficulty  index,  professors  were  counseled 
to  choose  a threshold  that  was  meaningful  and  appropriate  for  their  context.  Professors  were 
also  informed,  however,  that  an  item  should  not  have  a negative  discrimination  index. 

Evaluation  - For  the  2006  exams,  there  were  probably  more  poorly  discriminating  and 
negative  discriminating  items  than  would  be  desired.  As  indicated  above,  however, 
new  item  types  were  being  used  for  the  first  time  by  most  of  the  professors  and  they 
are  still  learning  how  to  use  item  analysis  data  to  improve  item  performance  for  future 
administrations. 
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Table  B.6.1:  2005  vs.  2006  Test  and  Test  Analysis  Comparisons  - Ibrahim  Abdel-Motaleb 


Midterm 

Test  Comparisons  Fall  2005  Fall  2006 


Based  on  Item  Bank 


Fall  2005 


Based  on  Table  of 
Specifications 

Overall  Appearance  - well 
laid  out;  pleasing 
appearance,  grammar, 
etc. 


Overall  Instructions  - 
clear,  unambiguous 


Instructions  for  Item 
Subsets  - clear, 
unambiguous 


Number  of  short,  discrete 
items  vs  longer  items 


Number  of  objectively 
scored  vs  subjectively 
scored  items 


Item  Quality  - clear, 
direct,  well  written,  no 
clues 


OK.  Test  is  only 
one  item,  3 parts, 
20  pts. 


Not  reaiiy  there. 
Only  one  item. 


One  long  item  - 20 
pts. 


Item  is  subjectively 
scored.  No  scoring 
criteria  provided  to 
students. 


Appears  OK. 


Very  good!  33 
Items. 


Not  much  there. 


Yes.  Some  for 
matching  and 
others. 


Test  contains 
mostly  short, 
discrete  items 
rather  than  a single 
long  item. 

Several  MC  items 
(objectively  scored) 
and  short  answer 
items  (subjectively 
scored). 


Appears  OK. 


Take  home  exam. 
One  item,  3 parts, 
50  pts. 

Instructions  may  be 
satisfactory.  But  pt 
values  for  items  & 
overall  test  not 
specified. 


One  long  item,  3 
parts,  50  pts. 


Subjectively 
scored.  No  scoring 
criteria  provided. 


Appears  OK. 


Item  Analysis 
Comparisons 


Item  Difficulty  - number 
of  very  difficult  Items 


One  item,  3 parts, 
20  pts. 

diff  - 67%,  the  3 
parts  not  analyzed 
separately. 


33  items  but  only 
20  in  item  analysis 
(?) 

Diff:  16  items  > 
50% 

4 items  < 50% 


One  item,  3 parts, 
50  pts 

Lowest  diff  = 72%. 
Overall  test 
average  = 77% 


Item  Discrimination  - 
number  of  poorly 
discriminating  Items 


Can't  compute 
since  analyzed  as 
one  item,  not  for 
three  parts. 


Disc:  2 items  0.0- 
.15 

1 item  negative. 


Lowest  disc  = .69. 
But  high  disc 
expected  since 
only  3 parts. 


Table  B.6.2:  2005  vs.  2006  Test  and  Test  Analysis  Comparisons  - Abul  Azad 


Midterm 

Final 

Test  Comparisons 

Fall  2005 

Fall  2006 

Fall  2005 

Fall  2006 

Based  on  Item  Bank 

No 

Yes 

No 

Yes 

Based  on  Table  of 
Specifications 

No 

Yes 

No 

Yes 

Overall  Appearance  - well 
laid  out;  pleasing 
appearance,  grammar, 
etc. 

Generally  good. 
Tests  called  First 
In-course  Exam 
and  Second  In- 
course  Exam. 
Total  of  18  Items. 

About  the  same  as 
F05;  called 
Midterm. 

Total  of  12  Items. 

Generally  good. 
Only  8 Items. 

Similar  to  F05. 

7 Items  (last  Item 
mislabeled  as  #8) 
10  parts:  la,  1b, 
1c,  etc. 

Overall  Instructions  - 
clear,  unambiguous 

No  overall  test 
Instructions. 

No  overall  test 
Instructions. 

No  overall  test 
Instructions. 

No  overall  test 
Instructions. 

Instructions  for  Item 
Subsets  - clear, 
unambiguous 

NA 

NA 

NA 

NA 

Number  of  short,  discrete 
items  vs  longer  items 

Combination  of 
short-answer  and 
longer-answer 
Items.  But  Items 
have  multiple  parts 
(la,  1b,  1c,  etc. 

Fewer  short-  and 
long-answer  Items. 
Five  MC  Items. 

Most  Items  appear 
to  be  longer, 
problem-type; 
remainder  are 
short-answer. 
Some  with  multiple 
parts  (la,  1b,  1c). 

All  Items  are  short- 
er long-answer 
type 

Number  of  objectively 
scored  vs  subjectively 
scored  items 

All  Items  appear 
subjectively 
scored. 

Five  of  the  twelve 
Items  are 

objectively  scored. 

All  are  subjectively 
scored. 

All  are  subjectively 
scored. 

Item  Quality  - clear, 
direct,  well  written,  no 
clues 

Items  appear  clear 
and  direct. 

Items  appear  clear 
and  direct. 

Items  appear  clear 
and  direct. 

Items  appear  clear 
and  direct. 

Item  Analysis 
Comparisons 

First  2 tests  - 18 
Items  = 200  pts;  12 
Items  & 6 Items 

12  Items  = 100  pts. 

8 Items  - 100  pts. 

1 0 Items/parts 
analyzed  100  pts. 

Item  Difficulty  - number 
of  very  difficult  items 

14  of  18  Items 
analyzed. 

All  Items  were  very 
easy:  smallest  diff 
= 72%. 

1 dIff  = 20% 

Very  easy  - 
smallest  diff  = 70%. 

Lowest  diff  = 54% 
Others  61  %-81% 

Item  Discrimination  - 
number  of  poorly 
discriminating  items 

3 Items  had 
negative 
discrimination 
Indices  - partly 
because  they  were 
so  easy  and  high 
scorers  got  them 
wrong. 

4 Items  disc  In  0.0- 
.15. 

1 Item  negative 
disc. 

2 Items  with  disc 
<.10 

No  negative 
discriminators. 

Lowest  disc  = .44 
Others  .71 -.94 
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Table  B.6.3:  2005  vs.  2006  Test  and  Test  Analysis  Comparisons  - Brianno  Coller 


Midterm 

Final 

Test  Comparisons 

Fall  2005 

Fall  2006 

Fall  2005 

Fall  2006 

Based  on  Item  Bank 

No 

Yes 

No 

Yes 

Based  on  Table  of 
Specifications 

No 

Yes 

No 

Yes 

Overall  Appearance  - well 
laid  out;  pleasing 
appearance,  grammar, 
etc. 

Not  really  a 
midterm;  just 
quizzes  #3,  #4  and 
#5.  15  pts  and  2 
problems  each 
quiz.  Total  45  pts. 

Great  looking 
exam! 

34  Items,  36  pts. 
Pts.  specified  for 
each  Item. 

Doesn't  really  have 
the  formal 
appearance  of  an 
exam.  4 problems, 
1 5 pts  each  = 60 
pts. 

Great  looking 
exam! 

33  Items,  33  pts. 
Pts.  specified  for 
each  Item. 

Overall  Instructions  - 
clear,  unambiguous 

NA 

Great  - 

Comprehensive 

None 

Good 

Instructions  for  Item 
Subsets  - clear, 
unambiguous 

NA 

Very  clear 

NA 

Very  clear 

Number  of  short,  discrete 
items  vs  longer  items 

4 short-answer 
problems,  2 MC 

All  are  short, 
discrete  Items 

4 long  problems. 

All  are  short, 
discrete  Items 

Number  of  objectively 
scored  vs  subjectively 
scored  items 

Only  2 Items 
objectively  scored. 

All  are  objectively 
scored 

All  subjectively 
scored. 

All  are  objectively 
scored 

Item  Quality  - clear, 
direct,  well  written,  no 
clues 

OK 

Very  good 

OK 

Very  good 

Item  Analysis 
Comparisons 

Each  quiz  analyzed 
separately  - not 
combined,  2 
problems  each. 

34  Items  analyzed 

4 problems 
analyzed 

33  Items  analyzed 

Item  Difficulty  - number 
of  very  difficult  items 

1 (of  6)  diff  <50% 
Oulz  means  are 
61%,  52%  and 
42% 

14  Items  dIff  < 50% 
Test  mean  = 55% 

1 problem  diff 
=50% 

Test  mean  = 58% 

9 Items  diff  < 50% 
Test  mean  = 59% 

Item  Discrimination  - 
number  of  poorly 
discriminating  items 

Al  6 disc  >.56 

2 Items  disc  0.0-.15 

AII4dlsc>.58 

3 Items  disc  0.0-.15 
1 Item  disc 
negative 
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Table  B.6.4:  2005  vs.  2006  Test  and  Test  Analysis  Comparisons  - Abhijit  Gupta 


Midterm 

Final 

Test  Comparisons 

Fall  2005 

Fall  2006 

Fall  2005 

Fall  2006 

Based  on  Item  Bank 

No 

Yes 

No 

Yes 

Based  on  Table  of 
Specifications 

No 

Yes 

No 

Yes 

Overall  Appearance  - well 
laid  out;  pleasing 
appearance,  grammar, 
etc. 

Not  much  here.  3 
problems.  Slightly 
difficult  to  read. 
Poor  photocopy? 

A much  better- 
looking exam  than 
F05. 

Could  be  better. 

Only  4 items. 
Poor  photocopy. 

Much  improved 
appearance  over 
F05. 

Overall  Instructions  - 
clear,  unambiguous 

OK 

Good.  Includes  pt. 
values  for  each 
item. 

OK 

Good,  with  pt 
values  for  each 
item. 

Instructions  for  Item 
Subsets  - clear, 
unambiguous 

NA 

Yes,  good. 

NA 

Very  good! 

Number  of  short,  discrete 
items  vs  longer  items 

All  3 items  are  long 
problem  type. 

All  but  3 items  are 
MC,  TF. 

41  total  items. 

All  4 are  long 
problem  type. 

All  but  4 items  are 
MC,  TF. 

30  total  items. 

Number  of  objectively 
scored  vs  subjectively 
scored  items 

All  subjectively 
scored. 

All  but  3 are 
objectively  scored. 

All  subjectively 
scored. 

26  items  are 
objectively  scored. 

Item  Quality  - clear, 
direct,  well  written,  no 
clues 

Probably  OK. 

Very  good. 

OK 

Very  good! 

Item  Analysis 
Comparisons 

3 items 

41  items  analyzed 

4 items 

30  items  analyzed 

Item  Difficulty  - number 
of  very  difficult  items 

Lowest  diff  = 53% 
Test  mean  = 67% 

22  items  with  diff 
<50%. 

Test  mean  = 43%. 
Hard  test! 

Lowest  diff=44% 
Test  mean  = 67% 

7 items  with  diff 
<50% 

Test  mean  = 60% 
Not  as  hard  as 
midterm. 

Item  Discrimination  - 
number  of  poorly 
discriminating  items 

All  disc  good  (>.50) 
Expected,  with  only 
3 items. 

8 items  disc  0.0-.15 
3 items  negative 
disc 

1 item  disc  =.17, 
others  >.60 

2 items  disc  0.0-.15 
3 items  negative 
disc. 
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Table  B.6.5:  2005  vs.  2006  Test  and  Test  Analysis  Comparisons  - Reinaldo  Moraga 


Midterm 

Final 

Test  Comparisons 

Fall  2005 

Fall  2006 

Fall  2005 

Fall  2006 

Based  on  Item  Bank 

No 

Yes 

No 

Yes 

Based  on  Table  of 
Specifications 

No 

Yes 

No 

Yes 

Overall  Appearance  - well 
laid  out;  pleasing 
appearance,  grammar, 
etc. 

OK 

Take  home? 
Only  2 problems, 
100  pts. 

Great  looking 
exam. 

25  Items,  100  pts. 

OK 

6 problems. 

5 from  textbook, 
100  pts.  Take 
home? 

Part  1 : 3 long 
problems  from  text 
= 100  pts. 

Part  2:  Great 
looking  exam.  25 
Items,  25  pts. 

Overall  Instructions  - 
clear,  unambiguous 

OK 

Good.  More 
comprehensive 

OK 

Good  for  part  2 

Instructions  for  Item 
Subsets  - clear, 
unambiguous 

NA 

NA 

NA 

NA 

Number  of  short,  discrete 
Items  vs  longer  Items 

Both  are  long 
problems. 

All  Items  are  short, 
discrete. 

All  are  long 
problems. 

25  Items  In  part  2 
are  short,  discrete. 

Number  of  objectively 
scored  vs  subjectively 
scored  Items 

Subjectively 

scored. 

All  are  MC  Items. 

Subjectively 

scored. 

3 In  part  1 are 
subjectively 
scored. 

25  In  part  2 are  all 
MC. 

Item  Quality  - clear, 
direct,  well  written,  no 
clues 

OK 

Appears  clear  and 
direct. 

OK 

Good. 

Item  Analysis 
Comparisons 

2 Items  analyzed 

25  Items  analyzed 

6 Items  analyzed 

28  Items  analyzed 

Item  Difficulty  - number 
of  very  difficult  Items 

Both  diff  around 
48% 

7 Items  dIff  < 50% 
Test  mean  = 55% 
(Hard  test) 

All  diff  > 65% 

9 Items  diff  < 50% 
Test  mean  = 60% 

Item  Discrimination  - 
number  of  poorly 
discriminating  items 

Both  disc  around 
.84 

1 Item  disc  0.0-.15 
(a  very  easy  Item) 
1 Item  negative 
disc. 

All  disc  > .46 

4 Items  disc  0.0- 
.15 

6 Items  disc 
negative 
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Table  B.6.6:  2005  vs.  2006  Test  and  Test  Analysis  Comparisons  - Regina  Rahn 


Midterm 

Final 

Test  Comparisons 

Fall  2005 

Fall  2006 

Fall  2005 

Fall  2006 

Based  on  Item  Bank 

No 

Yes 

No 

Yes 

Based  on  Table  of 
Specifications 

No 

Yes 

No 

Yes 

Overall  Appearance  - well 
laid  out;  pleasing 
appearance,  grammar, 
etc. 

Good 

5 items  with 
muitipie  parts. 
Totai  of  13 
items/parts. 

Great 

30  items,  100  pts. 

Good 

6 items  with 
multiple  parts. 
(Take  home?) 
Total  of  19 
items/parts 

Good. 

17  items,  105  pts. 

Overall  Instructions  - 
clear,  unambiguous 

No  overali  test 
instructions 

No  overali  test 
instructions 

No  overall  test 
instructions 

No  overall  test 
instructions 

Instructions  for  Item 
Subsets  - clear, 
unambiguous 

Good 

Good 

Good 

Good. 

17  items,  105  pts. 

Number  of  short,  discrete 
items  vs  longer  items 

Most  are  probiem- 
type. 

Some  may  be 
short-answer. 

5 short  answer 
25  MC,  TF, 
Matching 

Most  are  problem- 
type. 

Some  may  be 
short-answer. 

Most  are  short 
answer,  remainder 
are  longer  answer. 

Number  of  objectively 
scored  vs  subjectively 
scored  items 

Ali  are  subjectiveiy 
scored. 

5 subjectively 
scored 

25  objectively 
scored 

All  are  subjectively 
scored. 

All  are  subjectively 
scored. 

Item  Quality  - clear, 
direct,  well  written,  no 
clues 

Good 

Good 

Good 

Good 

Item  Analysis 
Comparisons 

13  items/parts 
anaiyzed 

30  items  analyzed 

6 items  analyzed- 
not  by  parts. 

17  items  analyzed 

Item  Difficulty  - number 
of  very  difficult  items 

2 items  diff  <50% 
Test  mean  = 70% 

3 items  diff  < 50% 
Test  mean  = 78% 

All  diff  >73% 
Test  mean  = 82% 

All  items  diff  > 50% 
Test  mean  = 78% 

Item  Discrimination  - 
number  of  poorly 
discriminating  items 

1 disc  < .15  (=.07) 

5 items  disc  0.0-.15 
2 items  disc 
negative 

1 disc  <.15  (=.06) 
(1  high  scorer  did 
poorly  on  this 
item.) 

1 item  disc  0.0-.15 
3 items  disc 
negative 
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Table  B.6.7:  2005  vs.  2006  Test  and  Test  Analysis  Comparisons  - Robert  Tatara 


Midterm 

Final 

Test  Comparisons 

Fall  2005 

Fall  2006 

Fall  2005 

Fall  2006 

Based  on  Item  Bank 

No 

Yes 

No 

Yes 

Based  on  Table  of 
Specifications 

No 

Yes 

No 

Yes 

Overall  Appearance  - well 
laid  out;  pleasing 
appearance,  grammar, 
etc. 

Courier  font. 

25  items  = 100  pts. 

Bigger,  more 
attractive  font. 
Better  overall 
appearance.  30 
items  = 30  pts. 

Courier  font. 
Layout  and 
appearance  OK. 
35  items  = 100  pts. 

Better  overall 
appearance. 
Better  font 

Overall  Instructions  - 
clear,  unambiguous 

Generally  OK 

Much  better, 
clearer,  more 
complete. 

OK 

Much  better, 
clearer,  more 
complete. 

Instructions  for  Item 
Subsets  - clear, 
unambiguous 

NA 

NA 

NA 

NA 

Number  of  short,  discrete 
items  vs  longer  items 

Some  items  are 
longer,  problem 
type.  Some  are 
short  answer. 

No  long  problem- 
type.  Fewer  short 
answer.  More  are 
MC,  TF,  matching. 

Several  short- 
answer.  Some 
longer  problem- 
type.  Not  quite  half 
are  MC,  TF,  etc. 

No  long  problem- 
type.  Fewer  short 
answer.  More  are 
MC,  TF,  matching. 

Number  of  objectively 
scored  vs  subjectively 
scored  items 

Most  are 
subjectively 
scored.  Small 
number  are 
objectively  scored 

Most  are 

objectively  scored  - 
MC,  TF,  matching. 
Very  few  are  short 
answer, 
subjectively 
scored. 

19  subjectively 
scored. 

16  objectively 
scored  - MC,  TF, 
etc. 

Most  are 

objectively  scored  - 
MC,  TF,  matching. 
Very  few  are  short 
answer, 
subjectively 
scored. 

Item  Quality  - clear, 
direct,  well  written,  no 
clues 

Perhaps  some 
slight  ambiguities  -- 
but  mostly  clear 
and  direct. 

Appears  clear  and 
direct 

Generally  good. 

Appears  clear  and 
direct 

Item  Analysis 
Comparisons 

25  items  analyzed 

30  items  analyzed 

35  items  analyzed 

50  items  analyzed 

Item  Difficulty  - number 
of  very  difficult  items 

7 items  (39%)  diff  < 
50% 

Test  mean  = 66% 

9 items  (43%)  diff  < 
50% 

Test  mean  62% 

4 items  (13%)  diff  < 
50% 

Test  mean  74% 

10  items  (20%)  diff 
< 50% 

Test  mean  72% 

Item  Discrimination  - 
number  of  poorly 
discriminating  items 

1 item  disc 
negative 

5 items  disc  0.0-.15 
4 items  disc 
negative 

10  items  disc  0.0- 
.15 

2 items  disc 
negative 

8 items  disc  0.0-.15 
4 items  disc 
negative 
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ABET/  NAIT  AND  STUDENT  LEARNING  OUTCOMES  SUMMARY 

(See  Reports  in  B.V.a.l  and  2;  also,  Professor  Examples  in  B.7.b.l-5;  also 

Portfolio  Section  A. 5 

Jule  Dee  Scarborough,  Ph.D. 

Upon  completing  the  overall  course  analysis,  teaching  models  and  styles  analyses,  the 
student  learning  style  analyses,  and  the  analyses  for  achieving  the  levels  of  Bloom’s 
Taxonomy  and  Dale’s  Cone  of  Learning,  the  professors  engaged  on  a new  interactive 
path  between  continuing  to  analyze  aspects  of  the  2005  course  and  beginning  the 
redevelopment  of  the  2006  course  for  the  experimental  research  semester. 

Process  for  Developing  the  2006  Course  Outcomes 

During  the  Student  Learning  Outcomes  program  component,  the  faculty  members 
reconsidered  their  course  from  the  student  learning  outcomes  perspective.  They  each 
determined  the  ABET  or  NAIT  standards  or  outcomes  that  were  important  for  their 
courses  and  redeveloped  student  learning  outcomes  at  the  primary,  second,  and  third 
levels  of  breakout.  Once  the  outcomes  were  developed  and  written  appropriately  as 
outcomes  oriented  statements  that  were  measurable  and  active,  they  put  them  into  a chart 
format.  In  preparation  for  the  chart,  most  of  them  first  developed  an  outline-like 
document  or  “list.”  Not  all  professors  were  proficient  at  the  American  or  English  concept 
of  “outlining”  showing  levels  of  content  breakout.  Some  opted  for  a list  approach  to  this 
activity.  Eor  the  most  part,  the  list  worked  to  serve  our  needs,  which  was  to  reveal  the 
complexity  of  the  primary  outcomes  and  break  them  out  into  second  and  third  level 
outcomes  so  the  professors  would  better  understand  the  complexity  of  the  content  in  the 
course.  This  process  also  served  to  reveal  the  embedded  general  education  knowledge 
and  skills  expected  of  students  (e.g.  mathematics,  science,  and  communications).  And 
even  though  the  program  leader  had  already  prepared  the  chart(  such  that  the  ABET  and 
NAIT  standards  or  outcomes  revealed  the  embedded  NIU  general  education  goals  written 
as  outcomes),  this  was  an  important  aspect  of  the  redevelopment  of  the  2005  course 
outcomes.  The  process  led  the  professors  into  a deeper  understanding  of  the  embedded 
content,  whether  second  and  third  level  engineering  and  technology  knowledge  and  skills 
or  general  education,  and  they  became  much  more  aware  of  what  they  were  expecting  of 
students  and  trying  to  achieve  with  student  learning.  We  accomplished  this  through  a 
variety  of  processes:  outlining,  lists,  matching,  analysis,  and  mapping  connections.  The 
charts  worked  well  to  assist  in  the  process  and  organize  the  professors’  redevelopment  of 
the  2005  course  outcomes. 

Once  the  student  learning  outcomes  were  developed  and  connected  to  the  national 
standards  or  outcomes,  ABET  or  NAIT,  and  the  embedded  general  education  knowledge 
and  skills  were  identified,  the  professors  analyzed  their  new  student  learning  outcomes  to 
determine  which  of  Bloom’s  Knowledge  Dimensions  were  represented  by  the  course 
content  (e.g.  factual,  conceptual,  procedural,  or  meta-cognitive).  They  then  analyzed  the 
new  outcomes  to  determine  or  confirm  that  the  outcomes  were  written  to  achieve  more  of 
the  upper  levels  of  Bloom’s  Cognitive  Process  Dimension,  using  either  the  traditional  or 
revised  version  (e.g.,  traditional:  knowledge,  comprehension,  application,  analysis, 
synthesis,  and  evaluate;  revised:  remember,  understand,  apply,  analyze,  evaluate,  and 
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create).  Finally,  the  professors  analyzed  the  outcomes  for  level  of  active  versus  passive 
learning  on  Dale’s  Cone. 

Performance 

Transitioning  from  analysis  of  the  2005  courses  to  their  redevelopment,  we  will  now 
speak  in  terms  of  the  development  of  the  2006  version  of  their  2005  courses.  The 
professors  performed  extremely  well  in  the  development  of  the  2006  courses  and  the 
student  learning  outcomes  that  define  the  courses.  The  learning  outcomes  statements 
were  expressed  more  appropriately  with  active  verbs  and  as  measurable  outcomes  and 
were  more  specific.  The  terminology  served  to  more  clearly  present  what  was  to  be 
learned  by  the  students,  and  the  learning  statements  were  more  coherent.  With  the 
breakout  of  the  primary  statements  into  the  second  and  third  levels,  it  became  much 
easier  to  understand  what  was  to  be  accomplished  in  the  courses  and  by  students. 
Furthermore,  the  overt  identification  and  mapping  of  the  embedded  general  education 
outcomes  clearly  revealed  the  expectations  related  to  the  underlying  mathematics, 
science,  and  communication  foundations  of  the  engineering  and  technology  knowledge 
and  skill  content.  Using  Bloom’s  Knowledge  and  Cognitive  models  served  to  reveal  and 
assure  the  professors  that  the  outcomes  statements  were  capable  of  formally  achieving  the 
upper  cognitive  levels  and  inherent  higher  levels  of  critical  thinking.  Identifying  the 
content  by  Bloom’s  Knowledge  Dimensions  served  to  confirm  that  the  content  was 
addressing  more  than  factual  knowledge  (i.e.,  lower  levels  of  the  Cognitive  Process 
Dimension).  They  tried  to  more  formally  show  that  the  four  (facts,  conceptual, 
procedural,  and  metacognitive  knowledge)  were  represented  in  the  student  learning 
outcomes.  The  more  knowledge  dimensions  represented  in  course  content,  the  greater  the 
possibilities  for  learning  to  achieve  the  higher  levels  of  the  cognitive  process  dimensions. 
This  one  aspect,  however,  was  not  accomplished  as  deeply  as  we  would  have  preferred. 
The  analysis  related  to  Bloom’s  Knowledge  Dimension  needed  more  time  and  focus. 
However,  most  of  the  professors  understood  how  it  could  help  them  analyze  their  content 
and  improve  its  coverage  if  they  included  content  from  more  of  the  knowledge 
dimensions,  realizing  that  if  they  are  addressing  each  of  Bloom’s  Knowledge 
Dimensions,  they  are  more  likely  to  achieve  the  higher  levels  of  Bloom’s  Cognitive 
Process  Dimension  more  often.  They  became  aware  of  the  potential  of  the  knowledge 
dimension,  but  during  our  process  did  not  use  it  to  its  fullest  power  to  improve  the 
content  of  their  courses.  However,  their  course  contents  did  reflect  knowledge  from 
across  Bloom’s  knowledge  dimensions;  but  it  was  documented  as  completely  as  possible. 
Finally,  the  professors  determined  whether  each  outcome  was  active,  passive,  or 
intermediately  active  on  Dale’s  Cone  of  Learning.  This  assisted  them  to  focus  on  the 
quality  of  learning  through  purposeful  constructivism,  where  the  burden  of  learning  to 
build  knowledge  and  skills  through  accomplishing  real  world  tasks  is  on  the  student, 
going  beyond  the  more  abstract  memorization  and  limited  comprehension  levels  to  being 
able  to  use  knowledge,  manipulate  it,  extend  it  through  problem  based  learning  and 
across  different  learning  contexts.  The  professors  tried  to  build  scaffolding  into  the 
learning  process  to  increase  depth  and  understanding  through  increased  ability  to  use 
knowledge  and  skills  being  learned  while  also  engaged  in  learning. 
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Results 

Overall,  the  2006  courses  have  greatly  improved  student  learning  outcomes  that  will 
engage  students  in  learning  more  actively  at  the  upper  levels  of  Bloom’s  Cognitive 
Process  Dimension.  The  professors  are  also  cognizant  of  Bloom’s  Knowledge 
Dimensions  and  that  they  should  strive  to  address  the  four  types  of  knowledge  with 
particular  purpose.  They  did  check  that  the  knowledge  and  skills,  etc.  represented  in  the 
student  learning  outcomes  crossed  the  four  knowledge  dimensions.  However,  they  could 
have  used  more  time  to  focus  on  this  aspect  of  our  program.  The  professors  benefited 
from  using  Dale’s  Cone  to  gauge  the  quality  of  active  learning.  Ultimately,  Bloom  and 
Dale’s  models  have  become  informal  and  formal  metrics  for  continuous  improvement  of 
the  courses,  teaching,  assessments,  and  student  learning.  The  professors  have  a much 
deeper  understanding  of  what  outcomes  statements  should  achieve,  their  connection  to 
the  ABET  and  NAIT  standards  and  outcomes,  their  relationship  to  the  embedded  general 
education  goals,  and  that  they  are  written  to  lead  students  to  “knowing  about,” 

“knowing,”  or  “being  able  to  “do”  or  perform.”  They  also  better  understand  what  active 
or  engaged  learning  means  and  that  if  they  are  to  lead  students  to  “using”  knowledge  and 
skills,  then  active  learning  involves  creating  a different  type  of  learning  environment:  one 
that  is  inquiry  and  discovery  driven  and  one  that  goes  beyond  “problem  solving”  to 
problem  based  learning  where  students  have  the  opportunity  to  perform  real  world  tasks 
using  their  knowledge  and  skills  and  where  expectations  for  performance  are  clearly 
understood  by  professors  and  students  because  there  are  clear  and  well  defined 
performance  standards  and  criteria.  They  understand  how  to  use  the  Bloom’s  models  to 
analyze  and  develop  their  courses.  Of  great  importance  is  that  professors  now  see  the 
embedded  general  education  outcomes  as  more  than  prerequisites  or  foundations  for 
learning  the  engineering  and  technology  content.  The  professors  now  better  understand 
that  the  engineering  and  technology  learning  context  and  content  continues,  extends, 
deepens,  and  expands  the  learning  of  general  education  knowledge  and  skills  and, 
ultimately,  that  they  are  responsible  for  continuing  the  learning  of  mathematics,  science, 
and  communication  at  higher  levels  in  the  engineering  and  technology  contexts.  That  is  a 
very  different  viewpoint  and  understanding.  The  following  data  tables  support  that 
conclusion. 

In  summary,  the  outcomes  can  continue  to  improve  and  be  refined,  but  the  learning, 
demonstrated  by  the  professors  through  their  2005  outcomes  analyses  and  newly 
developed  2006  outcomes,  which  were  greatly  improved,  revealed  their  deeper 
understanding  and  ability  to  make  course  content  decisions  and  to  use  a stronger 
theoretical  basis  for  making  those  decisions.  They  have  the  knowledge  and  skill  to 
continue  this  process  from  this  point  forward,  as  they  have  clearly  demonstrated 
their  significant  gain  in  knowledge,  skills,  and  ability  regarding  the  development  of 
student  learning  outcomes.  The  Outcomes  program  component  led  to  the  Student 
Assessment  program  component. 

See  Copy  of  Section  B.5.b  GAPS  Analysis  (Discussion  of  Student  Learning 

Outcomes)  Below 
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GAPS  ANALYSIS  SUMMARY  (FALL  2005  AND  FALL  2006) 

Jule  Dee  Scarborough,  Ph.D. 

Student  Learning  Statements  ( Outcomes ) 

In  the  initial  analysis  of  the  Fall  2005  courses  - where  we  began,  professors  used  their 
existing  course  syllabi.  Although  as  a college,  we  had  improved  our  student  learning 
statements  during  the  accreditation  process,  they  remained  rather  unorganized  and  weak 
in  content  and  appropriate  expression.  The  learning  statements  were  expressed  in  mixed 
modes  across  syllabi.  Some  learning  statements  were  written  as  course  objectives;  others 
were  written  as  student  learning  objectives;  yet  others  were  written  as  more  outcome- 
oriented  statements.  However,  in  generalizing,  many  and  sometimes  most  of  the  student 
learning  statement  formats  across  syllabi  were  not  active,  clear,  measurable,  or  clearly 
outcomes-oriented,  where  the  professor  and  student  could  ascertain  exactly  what  was 
expected  and  would  be  measured,  and/or  determine  the  culminating  grade.  Three 
professors  expressed  the  learning  statements  in  a way  where  students  could  see  that  there 
was  a relationship  between  student  learning  outcomes  and  the  ABET  or  NAIT  outcomes, 
but  if  the  ABET  or  NAIT  outcomes  were  identified  by  a letter  and  not  stated,  then  the 
relationship  was  not  clear  nor  were  students  about  to  review  the  accreditation  outcomes 
for  their  own  information.  Two  professors  expressed  the  statements  more  clearly,  with 
written  statements  for  both  the  national  standards  and  the  learning  outcomes  for  the 
course.  The  other  five  professors  did  not  show  the  national  statements  in  narrative  but 
rather  identified  them  by  letter  or  number,  regarding  the  level  of  coverage  and  depth  of 
relationship.  This  had  little  meaning  for  students  and  did  not  make  it  easy  for  the 
professors  to  clearly  be  assured  of  direct  links  and  relationships.  Generally,  the 
statements  did  link  to  the  ABET  or  NAIT  standards  or  outcomes,  but  often  not  clearly  or 
strongly.  It  would  have  been  difficult  to  determine  a direct  link,  especially  in  light  of  the 
student  learning  assessments  being  used  for  the  2005  course.  Therefore,  we  examined  the 
2005  syllabi  and  course  content  related  to  the  standards  as  well  as  we  could,  with  the 
understanding  that  the  student  learning  outcomes  to  be  redeveloped  would  better  and 
more  clearly  and  directly  link  to  the  national  standards  and  assessments  - a two-way  link 
revealing  the  critical  knowledge  and  skill  connections. 

Below  are  two  charts  that  broadly  identify  the  standards  addressed  in  the  Eall  2005 
courses,  according  to  the  content  and  syllabus  analysis  by  each  professor  of  his/her 
course.  The  data  are  presented  (in  black)  as  collapsed  across  either  all  engineering 
professors  or  engineering  technology  professors  as  a broad  viewpoint.  The  Fall  2006 
courses  are  presented  in  red,  and  although  there  are  minor  differences  in  the  number  of 
standards  addressed,  there  is  a great  and  very  significant  difference  in  the  quality  of  the 
learning  statements  and  their  direct  links  to  both  the  national  standards  and  the  learning 
assessments.  The  tables  also  reflect  the  number  of  learning  outcomes  for  each  standard  by 
professor,  2005-2006  when  possible.  For  the  2006  courses,  the  professors  not  only  have 
improved  wording  and  expression,  but  the  knowledge  and  skill  connections  are  much 
stronger.  In  addition,  the  outcome  statements  are  improved  because  they  are  broken  out 
into  primary,  second,  and  third  level  statements.  The  quality  is  improved  not  only 
because  of  better  wording,  but  also  because  they  now  better  understand  the  difference 
between  complex  statements,  where  there  is  a cluster  of  outcomes  inherent  to  a single 
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primary  outcome  statement.  Thus  the  course  content  or  the  knowledge  and  skills  to  be 
taught  became  more  obvious  in  the  inherent  breakouts  of  second  and  sometimes  third 
level  outcomes.  This  provided  insight  and  assisted  the  professors  in  understanding  what 
underlying  or  inherent  knowledge  and  skills  were  required  for  a complex  cluster  of 
difficult  primary  learning  outcomes  - in  other  words,  the  knowledge  and  skills  inherent  to 
a single  complex  primary  outcome.  Therefore,  readers  may  be  amazed  at  the  number  of 
changes  that  resulted. 

Usually,  the  primary  statements  would  be  used  on  syllabi  or  other  reporting  documents, 
but  the  analysis  and  breakout  of  second  or  third  level  learning  statements  provided  a great 
learning  experience  for  professors  and  led  them  to  design  and  then  engage  students  in 
more  intentional,  thoughtful,  and  higher  quality  learning  experiences.  This  analysis  and 
process  can  lead  to  more  astute  teaching  and  student  learning,  student  assessments, 
instructional  choices,  learning  process  decisions,  and  more.  Remember,  each  course  is 
not  required  to  address  every  national  standard  or  outcome,  but  instead  the  standards  or 
outcomes  of  focus  selected  should  be  addressed  well.  It  is  important  that  the  professors 
understand  individual  course  versus  program  requirements,  that  there  is  a cumulative 
effect  across  courses  for  the  entire  program  and  that  the  overall  program  is  required  to 
address  all  national  standards  or  outcomes,  not  any  single  course;  therefore,  many 
standards  will  be  addressed  across  multiple  courses.  However,  particular  standards  may 
be  addressed  in  only  one  or  two  courses  across  the  program,  depending  on  content,  depth, 
program  level,  (e.g.,  introductory  or  capstone  course).  Professors  sometimes  mistakenly 
strive  to  address  all  or  too  many  outcomes;  thus  the  course  content  can  become  weak  or 
superficial.  Finally,  when  identifying  the  objectives  or  outcomes  listed  below,  an  * is 
used  where  one  objective  or  outcome  covers  more  than  one  ABET  outcome  or  NAIT 
standard  or  where  there  is  a greater  total  of  “Is”  than  the  total  in  the  number  in 
parentheses  (4).  The  determining  factor  is  the  level  of  coverage  of  content. 

Regarding  outcomes,  it  is  important  to  note  that  the  professors  analyzed  the  engineering 
or  technology  course  content  for  embedded  NIU  General  Education  Goals.  This  analysis 
led  them  to  more  deeply  understand  why  students  fail  to  perform  well  in  their  courses  if 
they  do  not  come  to  the  course  with  the  appropriate  general  education  knowledge  and 
skills  that  are  the  underlying  foundation  for  the  engineering  and  technology  content.  The 
program  leader  revealed  the  strong  relationships  between  NIU  General  Education  Goals 
(outcomes)  and  the  ABET  and  NAIT  standards  or  outcomes  by  aligning  and  inserting 
them  into  a worksheet.  That  made  it  much  easier  and  more  efficient  for  the  professors  to 
see  the  direct  relationships,  to  consider  the  importance  of  acknowledging  the  embedded 
general  education  goals/outcomes  as  part  of  their  course  content,  and  to  realize  that  even 
though  they  are  teaching  engineering  or  technology  courses,  they  are  actually 
concurrently  continuing,  extending,  expanding,  and  deepening  the  learning  of  general 
education  content  in  the  context  of  engineering  and  technology.  This  was  extremely 
important.  Our  professors  intuitively  knew  this  but  had  never  “studied”  the  connections, 
mapped  the  connections,  or  included  the  general  educational  goals  aligned  beside  their 
engineering  or  technology  outcomes.  They  had  also  never  thought  of  themselves  as 
continuing  the  learning  of  the  general  education  knowledge  and  skills  in  engineering  and 
technology  content.  They  considered  the  general  education  math,  science,  and 
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communication  knowledge  and  skills  as  prerequisites  and  only  dealt  with  them  when 
students  did  not  have  the  knowledge  or  skills  needed  to  perform  on  the  engineering  and 
technology  content.  Now  the  professors  understand  that  they  actually  continue  the 
learning  of  the  mathematics,  science,  eommunieation,  ete.  eontent  in  the  engineering  and 
teehnology  eontext.  The  ehart  below  reflects  the  2005  eourse  in  blaek  and  the  ehanges  for 
the  2006  course  in  red.  The  professors  improved  the  outcomes  and  connections  and  are 
committed  to  greater  depth  of  change  for  the  future.  This  was  a very  successful  program 
component,  resulting  in  signifieant  learning  and  change. 

Assumptions 

Beware  of  assumptions  when  scanning  the  chart  helow  and  noting  that  one  or  more 
course  outcome  numbers  did  not  seem  to  change.  For  example,  one  professor’s 
number  of  outcomes  did  not  change  from  2005-2006;  however,  the  quality  of  the 
outcomes  for  2006  was  very  significantly  different  and  improved.  Also  that 
professor’s  four  primary  outcomes  were  broken  out  into  second  and  third  level 
outcomes.  Again,  for  example,  one  primary  outcome  inherently  encompassed  five 
secondary  outcomes,  with  each  of  those  broken  out  into  a third  level.  Thus,  the 
quality  in  content,  linkages,  and  assessments  was  dramatically  different  and  greatly 
improved  for  most  of  the  courses. 
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Table  B.7.1:  Standards  ABET-Engineering  Outcomes 

(Eall  20( 

35  and  Eall  2( 

306  courses)  (5  engineering  professors) 

a.  apply 

b.  design/ 

C. 

d. 

e. 

f. 

g- 

h. 

i. 

j- 

k. 

math, 

conduct 

design  system. 

function  on 

identify,  formulate. 

understand 

ability  to 

understand  impact 

recognition  of 

knowledge  in 

ability  to  use 

science, 

experiments; 

component. 

inter- 

solve  engineering 

professional. 

communicate 

eng.  Solutions  on 

need  for,  and 

contemporary 

techniques. 

engineering 

analyze, 

process-given 

disciplinary 

problems 

ethical 

effectively 

global,  economic. 

ability  to 

issues 

skills,  and 

interpret  data 

constraints. 

teams 

responsibility 

environment,  society 

engage  in  life- 

modern 

etc. 

long  learning 

engineering 

tools 

Eall  2005  and  Eall  200 

6 Courses  - ABET  Outcomes 

5 + 

2+ 

4 + 

1-H 

4-1- 

1-H 

1-H 

1-H 

2-h  1-h 

4-h 

4-h 

Ipartial 

1+partial 

1-H  (written 

1 

(to  small 

1-H 

(no 

(could  do 

INR 

reports  only) 

INR 

effect) 

(students  don’t 

DOE) 

lots  more) 

INR 

2c 

use  unless 
asked  to) 

INR 
1 c 

1 c 

3c 

Ic 

2c 

3c 

Ic  (minor) 

2c 

Ic 

5+ 

2+ 

4+ 

2+ 

5+ 

None 

3-1- 

1-1- 

2-1- 

4-1- 

5-1- 

(6-11)  1-4 

1-1 

1-  2 

1-1 

*-8 

1-1 

1 - none 

* - none 

* . 2 

* -3 

(3/6-5)  1-5 

1-1 

1-5 

* - 1 

*-l 

1-1 

(4-4)  1 - 

1 

* -none 

1 - none 

* - none 

1-1 

* - none 

* - 1 

❖ 

❖ 

1-1 

(4-5)  1 - 5 

1-5 

3 

1-5 

* - none 

1-3 

* - none 

*- 1 

* - 1 

1-2 

(4-5)  1-2 

1-1 

1-1 

* - none 

1 - 1 

1-1 

Legend:  + = yes-okay;  c = need  to  consider;  other  notes 
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Table  B.7.2:  Standards  ABET/TAC/NAIT-Engineeiing  Technology  & Industrial  Technology 


(2  engineering  technology /tec 

inology  professors) 

(Eall  2005  and  Eall  2006  courses) 

a. 

b. 

c.  ability  to 

d. 

e. 

f. 

g- 

h. 

i. 

j- 

k. 

1. 

m. 

n. 

o. 

P- 

mastery  of 

ability  to 

conduct, 

ability  to 

ability  to 

ability  to 

ability  to 

ability 

recognize 

ability 

respect  for 

commit 

ability  to 

ability 

ability 

ability 

knowledge, 

apply 

analyze. 

apply 

function 

identify. 

commu- 

to  com- 

need  for. 

to 

diversity; 

to  quality. 

program 

to  use 

to 

to 

techniques, 

current 

interpret 

creativity 

effectively 

analyze. 

nicate 

municate 

ability  to 

under- 

know- 

timeliness. 

computers 

modern 

manage 

design. 

skills. 

knowledge; 

experi- 

in  design 

on  teams 

solve 

effectively 

effectively 

engage  in 

stand 

ledge  of 

continuous 

and/or  use 

labor- 

proiects 

mani- 

modern  tools 

adapt  to 

ments; 

of  systems, 

technical 

writing 

orally 

lifelong 

profes- 

contempor 

improve- 

computer 

atory 

effect- 

pulate, 

emerging 

apply 

components. 

problems 

learning 

sional. 

ary 

ment 

application 

tech- 

tively 

manage 

applications 

experiment 

processes 

ethical. 

profession- 

s 

niques. 

industri 

of  math. 

al  results  to 

social 

al,  societal. 

effectively 

skills, 

al 

science, 

improve 

responsi 

global 

equip- 

systems 

technology 

processes 

bilities 

issues 

ment 

effect- 

q- 

tively 

ability 

manage 
or  lead 

person- 

nel 

effect- 

ively 

Eall  2005  and  Eall  2006  Courses  - ABET/TAC/NAIT  Outcomes 

2+ 

2+ 

1+ 

1+ 

1+ 

2+ 

1+ 

1+ 

1+ 

1+ 

1-1- 

2+ 

2c 

1+ 

1-1- 

p- 

1 no 

re- 

Ic 

Ic 

Ic 

Ic 

Ic 

Ic 

Ic 

Ic 

Ic 

Ic 

sponse 

Ic 

None 

2+ 

2+ 

2+ 

2+ 

None 

2+ 

2+ 

None 

2+ 

1-1- 

1-1- 

1+ 

1+ 

1-1- 

None 

9 

not 

q- 

Ino  re- 

sure 

sponse 

Ic 

None 

P- 

(5-6) 

* -NR 

1-6 

1-5 

1- 1 

* - 1 

1-4 

6 

* . 4 

2 

1 

2 

1 

1-1 

NR-NR 

q- 

(6-19) 

*-NR 

1-10 

1-5 

*-5 

* -3 

1-6 

1 

4 

1-one 

1 

*-none 

^"none 

NR-NR 

Legend:  + = yes-okay;  c = need  to  consider;  other  notes 
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Table  B.7.3:  NIU  General  Education  Goals 


(Fall  2005  and  Fall  2006) 


(7  professors  across  engineering  and  technology) 


Writing 

Speaking 

Listening 

Quantitative 

Reasoning 

Use  of 

Resources- 

Technology 

Historical 
Development 
Of  Culture 

Significance 
of  Arts 

Cultural 

Traditions 

Philosophical 

Ideas 

Methods  in 
Science 
Methods  in 
Social  Science 

Interrelatedness 

Across 

Disciplines 

Social 

Responsibility 

Citizenship 

C C+ 

Earlier  it  was 
only  lab 
reports.  In 
fall  06,  they 
had  to  write 
reports  for 
three  PA 
tasks. 

C C+ 

Presentation 
of  PA  tasks 

+ + 
Listen  to 
guest  speaker, 
professor, 
fellow 
students 
during  PA 
task 

presentations 

+ C+ 

Homework, 

exams,  PA 

tasks  - all 

involved 

quantitative 

reasoning 

+ + 
In  addition 
to  labs  that 
required 
using  many 
resources, 
had  to  use 
outside 
resources 
for  PA 
tasks. 

NR  NA 

NR  NA 

NR  NA 

NR  NA 

NRC 

May  consider 
more  interaction 
with  EE  for 
signal  processing 

NR  C+ 

Discussed  issues 
such  as  energy 
conservation, 
noise,  pollution, 
ethics,  etc. 

C+ 

Did  consider 
and  add,  still 
needs 

improvement; 
will  keep 
working  on  it 

C 

9 

C+ 

Strong,  but 
could  be  better 

+ 

I’m  quite 
pleased 

NA 

NA 

C- 

C+ 

C 

C 

Ok  + 

Project, 

exams, 

homework, 

using  MS 

Word 

C + 

Oral 

presentations 

with 

PowerPoint 

c c- 

Ok  + 

Material 
requires  this 

Ok  + 

Software 

and 

computer  to 

solve 

problems 

NR  NA 

NR  NA 

NR  NA 

C NA 

Ok,  C+ 

Examples, 
exercises  with 
topics  from 
other  disciplines 

C C- 

+ + 
PAs  and 
homework 

+ + 
Pas  and 
discussion 
sessions 

+ + 
Lectures,  case 
studies, 
discussion 
sessions 

+ + 
Problem 
solutions, 
homework, 
midterm  and 
final  exams 

+ + 
PAs  and 
homework 

C NA 

C NA 

C NA 

+ + 
Problem 
solutions, 
homework, 
midterm,  final 
exams,  and  PAs 

+ + 
Case  studies  and 
PAs 

+ C- 

C - + well 

addressed 
through  PA 
reports 

+ NA- 

possible  to 
include  for 
future  course 

C + 

lectures 

+ + 
addressed  in 
project  design 
decisions 

C + 

well 

addressed 

through 

project 

design 

decisions 

NR  NA 

C + 

addressed 
through 
project  design 

NR  - NA 

C + 

well  addressed 
through  project 
design 

C C+ 

to  some  extend 
when  making 
design  decisions 

c c- 

possible  to 
include  for 
future  courses 

+ + 

Performance 

Tasks 

+ + 
Group 
learning  and 
interactions 

+ + 

Group 

interactions 

+ + 
Performance 
tasks,  lab 
demonstrations 

C NA 

+ C+ 

C + 

Lectures 

+ C+ 

+ NA? 

C C+ 

+ C+ 

C + 

C C- 

C C- 

+ N+ 

+ + 

+ NR 

C NA 

C NA 

+ NA 

C NA 

C N\A 

Legend:  + addressed  well;  NA-does  not  really  apply  in  professor’s  opinion;  C-  do  not  do  it,  but  still  need  to  consider  adding  it  in  as  professor  continues  to 
make  changes;  C+  did  consider  and  add  in;  still  needs  improvement  and  professor  will  keep  working  to  improve  or  add; 
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Table  B.7.a.l.l:  Engineering  Student  Learning  Outcomes  & Bloom  & Dale  (Fall  2005  and  Fall  2006) 

The  professors  analyzed  their  student  learning  outcomes  against  Bloom’s  Learning  Dimensions  and  Dale’s  Cone  of  learning.  The  analysis  of  the  2005  course  is 
presented  in  black  below  and  also  as  a composite,  number  of  outcomes  achieving  what  level  on  Bloom’s  and  Dale’s  models.  The  2006  course  analysis,  however,  is 
presented  in  red.  Dale’s  levels  are  presented  by  number  of  outcome  and  level  of  the  Cone.  For  Bloom,  each  outcome  is  listed  at  the  level  achieved.  *(1),  (2),  (3),  (4), 
(7)  ABET  Engineering  courses. 


Student  Learning  Outcomes  & Bloom  & Dale  (Fall  2005  and  Fall  2006) 


(7  professors  across  engineering  and  technology) 


# Outcomes: 

Dale's  Cone 

Knowledge 

Comprehension 

Application 

Analysis 

Synthesis 

Evaluate 

Critical 

05  reported  as  composite 

Levels  : 

Remember 

Understand 

Apply 

Analyze 

Evaluate 

Create 

Thinking  Level: 

06  reported-  speeific  outeome 

PA  A+ 

LM  H 

*(1)  6 outcomes  composite 

NR 

1+,  5NR 

2+,  4NR 

2+,  4NR 

Ic,  5NR 

1+,  5NR 

6NR 

2Lm  3M,  IH 

1-11  numbers  1-11 

1-11 

(8-10)  A-i- 

1,  3,  7.  8.  9 

2,  5,  6,  7. 10, 11 

1,4 

*(2)  3/6 

3P,  lA 

6-1- 

6-1- 

3-I-,  3c 

5c,  INR 

6c 

6c 

2L,  2L+,  IL/M,  IM; 

2A- 

1-3/6 

2Mc 

5 

1-5 

(10)  A-i- 

*(3)  2 

2PA-C 

2-i-c 

2-i-c 

2-I-C 

2-I-C 

2-I-C 

2c 

2Mc 

2 

1-2 

(8-10)  A-I- 

1,2 

*(4)  4 

2A,  2A-I- 

4-1- 

4-1- 

4-1- 

4-1- 

Ic,  l-i--,  2-1- 

4c 

2M,  2H 

5 

NR 

NR 

NR 

NR 

NR 

NR 

NR 

*(5)  5 

2P,  2A,  1A+ 

2+,  3NR 

2NR,  3+ 

2NR,  3+ 

NR 

NR 

NR 

3M 

19 

1(6); 

2-3  (9) 
4(6) 

5-19 

(9-10)A-i- 

NR 

NR 

NR 

NR 

NR 

NR 

4P,2A 

*(6)  6 

3(2) 

2-I-,  4c 

3-I-,  3c 

5-I-,  Ic 

4-I-,  2c 

5-I-,  Ic 

4c,  2-1- 

3L,  2M,  IH 

6 

4 (1,3,5) 
8-9  (1) 
8(1) 

1,  2,  3,  5 

4 

6 

IP,  IP-A, 

4-1- 

4-1- 

1-i-c,  3-1- 

Ic,  let-,  2-1- 

2c,  2-1- 

2c,  2-1- 

2c,  2c-i- 

*(7)  4 

2A-P 

1 

2 

3,4 

4 

1-4  (1-10) 

Bloom  (1956);  Anderson  & Krathwohl  (2001);  Legend  for  Blooms  levels:  NR  = no  response;  number  + = number  of  outcomes  at  that  level;  +c  = okay,  but  still  need  to  consider; 
c=need  to  consider  achieving;  c+=some  positive  accomplishment,  but  still  needs  work  (e.g.,  outcome  number  reported  by  each  Bloom  level) 

Dale  (1969):  Legend  for  Dale’s  levels:  9-10  =active  learning-doing  level;  8=active  learning-participating;  3-7=visual  receiving/passive;  2-l=verbal  receiving-passive, 

(e.g.,  outcome  number  - level) 
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ABET  Engineering  Outcomes 

Bloom's  Knowledge 
Dimension 

Dale’s  Cone 

Student 

Learning 

Objectives 

Bloom’s  Cognitive  Process  Dimension 

Passive 

Participating 

Active 

Knowledge 

Remember 

Comprehension 

Understand 

Applieation 

Apply 

Analysis 

Analyze 

Synthesis 

Evaluate 

Evaluate 

Create 

A.  Apply  knowledge  of  math, 
science,  engineering 

NIU  Gen  Ed  Goals  - 
Students: 

a.  Hi.  perform  basic 
computations,  display 
facility  with  use  of 
formal  and  quantitative 
reasoning  analysis  and 
problem  solving,  and 
interpret  mathematical 
models  and  statistical 
information. 

Factual  Knowledge 

Conceptual  Knowledge 

Procedural  Knowledge 

Meta-Cognitive 

Knowledge 

B.  Design  and  conduct 
experiments;  analyze  and 
interpret  data 

NIU  Gen  Ed  Goals  - Students: 
a.  Hi.  perform  basic 
computations,  display 
facility  with  use  of  formal 
and  quantitative  reasoning 
analysis  and  problem 
solving,  and  interpret 
mathematical  models  and 
statistical  information. 

Factual  Knowledge 

Conceptual  Knowledge 

Procedural  Knowledge 

Meta-Cognitive 

Knowledge 
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ABET  Engineering  Outcomes 

Bloom's  Knowledge 
Dimension 

Dale’s  Cone 

Student 

Learning 

Objectives 

Bloom’s  Cognitive  Process  Dimension 

Passive 

Participating 

Active 

Knowledge 

Remember 

Comprehension 

Understand 

Applieation 

Apply 

Analysis 

Analyze 

Synthesis 

Evaluate 

Evaluate 

Create 

C.  Design  a system,  component 
process  to  meet  desired  needs 
within  realistic  constraints  (e.g., 
economic,  environmental,  social 
political,  ethical,  health,  safety, 
manufacturability,  & 
sustainability). 

NIU  Gen  Ed  Goals  - 
Students: 

c.  develop  an  understanding 
of  the  relatedness  of  various 
disciplines  by  integrating 
knowledge  from  several 
disciplines  and  applying  that 
knowledge  to  an 
understanding  of  important 
problems  and  issues. 

9 

i, 

Factual  Knowledge 

Conceptual  Knowledge 

Procedural  Knowledge 

Meta-Cognitive 

Knowledge 
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ABET  Engineering  Outcomes 

Bloom's  Knowledge 
Dimension 

Dale’s  Cone 

Student 

Learning 

Objectives 

Bloom’s  Cognitive  Process  Dimension 

Passive 

Participating 

Active 

Knowledge 

Remember 

Comprehension 

Understand 

Applieation 

Apply 

Analysis 

Analyze 

Synthesis 

Evaluate 

Evaluate 

Create 

D.  Eunction  on  multi- 
disciplinary teams. 

NIU  Gen  Ed  Goals- 
Students; 
b.iii.  demonstrate  a 
knowledge  of  cultural 
traditions  and  philosophical 
ideas  that  have  shaped 
societies,  civilizations,  and 
human  self-conceptions, 
d.  develop  social 
responsibility  and 
preparation  for  citizenship 
through  global  awareness, 
environmental  sensitivity, 
and  an  appreciation  of 
cultural  diversity. 

Factual  Knowledge 

Conceptual  Knowledge 

Procedural  Knowledge 

Meta-Cognitive 

Knowledge 
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ABET  Engineering  Outcomes 

Bloom's  Knowledge 
Dimension 

Dale’s  Cone 

Student 

Learning 

Objectives 

Bloom’s  Cognitive  Process  Dimension 

Passive 

Participating 

Active 

Knowledge 

Remember 

Comprehension 

Understand 

Applieation 

Apply 

Analysis 

Analyze 

Synthesis 

Evaluate 

Evaluate 

Create 

E.  Identify,  formulate,  and  solv« 
engineering,  problems 

Gen  Ed  Goals  - Students: 
b.  develop  an  ability  to  use 
modes  of  inquiry  across  a 
variety  of  disciplines  in  the 
humanities  and  the  arts,  the 
physical  sciences  and 
mathematics,  and  social 
sciences. 

b.iv.  demonstrate  an  ability 
to  use  scientific  methods  and 
theories  to  understand  the 
phenomena  studied  in  the 
natural  and  social  sciences. 

Factual  Knowledge 

Conceptual  Knowledge 

Procedural  Knowledge 

Meta-Cognitive 

Knowledge 
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ABET  Engineering  Outcomes 

Bloom's  Knowledge 
Dimension 

Dale’s  Cone 

Student 

Uearning 

Objectives 

Bloom’s  Cognitive  Process  Dimension 

Passive 

Participating 

Active 

Knowledge 

Remember 

Comprehension 

Understand 

Applieation 

Apply 

Analysis 

Analyze 

Synthesis 

Evaluate 

Evaluate 

Create 

E.  Understand  professional  ai 
ethical  responsibility 

NIU  Gen  Ed  Goals  - 
Students: 
d.  develop  social 
responsibility  and 
preparation  for  citizenship 
through  global  awareness, 
environmental  sensitivity, 
and  an  appreciation  of 
cultural  diversity. 

id 

Factual  Knowledge 

Conceptual  Knowledge 

Procedural  Knowledge 

Meta-Cognitive 

Knowledge 

G.  Communicate  effectively 

NIU  Gen  Ed  Goals- 
Students: 

a.  develop  habits  of  writing, 
speaking,  and  reasoning 
necessary  for  continued 
learning. 

a.i.  communicate  clearly  in 
written 

English,  demonstrating 
ability  to 

comprehend,  analyze,  and 
interrogate  critically. 

Factual  Knowledge 

Conceptual  Knowledge 

Procedural  Knowledge 

Meta-Cognitive 

Knowledge 
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Bloom’s  Cognitive  Process  Dimension 

Knowledge  Comprehension  Application  Analysis  Synthesis  Evaluate 

Rememher  Understand  Apply  Analyze  Evaluate  Create 


ABET  Engineering  Outcomes 

Bloom's  Knowledge 
Dimension 

Dale’s  Cone 

Student 

Learning 

Objectives 

Bloom’s  Cognitive  Process  Dimension 

Passive 

Participating 

Active 

Knowledge 

Remember 

Comprehension 

Understand 

Applieation 

Apply 

Analysis 

Analyze 

Synthesis 

Evaluate 

Evaluate 

Create 

J.  knowledge  of  contemporary 
issues 

Factual  Knowledge 

Conceptual  Knowledge 

Procedural  Knowledge 

Meta-Cognitive 

Knowledge 

NIU  Gen  Ed  Goals  - 
Students: 
d.  develop  social 
responsibility  and 
preparation  for  citizenship 
through  global  awareness, 
environmental  sensitivity, 
and  an  appreciation  of 
cultural  diversity. 

K.  use  techniques,  skills,  and 
modern  engineering  tools 
necessary  for  engineering 
practice 

Factual  Knowledge 

Conceptual  Knowledge 

Procedural  Knowledge 

Meta-Cognitive 

Knowledge 

Note:  Additional  educational  outcomes  -NIU  General  Education  Goals  - articulated  by  the  overall  program 
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Table  B.7.a.2.1:  Engineering  Technology  and  Technology  Student  Learning  Outcomes  & Bloom  & Dale  (Fall  2005  and  Fall  2006) 


The  professors  analyzed  their  student  learning  outcomes  against  Bloom’s  Learning  Dimensions  and  Dale’s  Cone  of  Learning.  The  analysis  of  the  2005  course 
is  presented  in  black  below  and  also  as  a composite,  number  of  outcomes  achieving  what  level  on  Bloom’s  and  Dale’s  models.  The  2006  course  analysis, 
however,  is  presented  in  red.  Dale’s  levels  are  presented  by  number  of  outcome  and  level  of  the  Cone.  For  Bloom,  each  outcome  is  listed  at  the  level  achieved. 
* (5)  and  (6)  ABET/TAC/NAIT  courses. 


Student  Learning  Outcomes  & Bloom  & Dale  (Fall  2005  and  Fall  2006) 


(7  professors  across  engineering  and  technology) 


# Outcomes: 

05  reported  as  composite 

06  reported-  speeific 
outeome 

Dale's  Cone 
Levels  : P 
AA-h 

Knowledge 

Remember 

Comprehension 

Understand 

Application 

Apply 

Analysis 

Analyze 

Synthesis 

Evaluate 

Evaluate 

Create 

Critical 

Thinking  Level: 
LM  H 

(1)  6 outcomes 

NR 

1+,  5NR 

2+,  4NR 

2+,  4NR 

Ic,  5NR 

1+,  5NR 

6NR 

2Lm  3M,  IH 

composite 

1-11 

1,  3,  7.  8.  9 

2,  5,  6,  7. 10, 11 

1,4 

1-11  numbers  1-11 

(8-10)  A-i- 

3P,  lA 

6-1- 

6-1- 

3-I-,  3c 

5c,  INR 

6c 

6c 

2L,  2L+,  IL/M,  IM;  2Mc 

(2)  3/6 

2A- 

1-3/6 

1-5 

5 

(10)  A-i- 

2Mc 

2PA-C 

2-i-c 

2-i-c 

2-I-C 

2-I-C 

2-I-C 

2c 

(3)  2 

1-2 

1,2 

2 

(8-10)  A-I- 

2M,  2H 

2A,  2A-I- 

4-1- 

4-1- 

4-1- 

4-1- 

Ic,  l-i--,  2-1- 

4c 

(4)  4 

NR 

NR 

NR 

NR 

NR 

NR 

NR 

5 

3M 

2P,  2A,  1A+ 

2+,  3NR 

2NR,  3-1- 

2NR,  3+ 

NR 

NR 

NR 

*(5)  5 

1(6); 

19 

2-3  (9) 

NR 

NR 

NR 

NR 

NR 

NR 

4(6) 

5-19 

(9- 10)  A-I- 

4P,2A 

3L,  2M,  IH 

3(2) 

2-I-,  4c 

3-I-,  3c 

5-I-,  Ic 

4-i-,2c 

5-I-,  Ic 

4c,  2-1- 

*(6)  6 

4 (1,3,5) 

1,  2,  3,  5 

4 

6 

6 

8-9  (1) 

8(1) 

2c,  2c-i- 

IP,  IP-A, 

4-1- 

4-1- 

1-i-c,  3-1- 

Ic,  Ic-i-,  2-1- 

2c,  2-1- 

2c,  2-1- 

3,4 

2A-P 

1 

2 

(7)  4 

4 

1-4  (1-10) 

Bloom  (1956);  Anderson  & Krathwohl  (2001);  Legend  for  Blooms  levels:  NR  = no  response;  number  + = number  of  outcomes  at  that  level;  +c  = okay,  but  still  need  to  consider;  c=need  to  consider 
achieving;  c+=some  positive  accomplishment,  but  still  needs  work  (e.g.,  outcome  number  reported  by  each  Bloom  level)  Dale  (1969):  Legend  for  Dale’s  levels:  9-10  =active  learning-doing  level;  8=active 
learning-participating;  3-7=visual  receiving/passive;  2-l=verbal  receiving-passive  (e.g.,  outcome  number  - level) 
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ABET/TAC/NAIT  Engineering 
& Technology  Outcomes 

Bloom's 

Knowledge 

Dimension 

Dale’s  Cone 

Student  Learning 
Objectives 

Bloom’s  Cognitive  Process  Dimension 

Passive 

Participating 

Active 

Knowledge 

Remember 

Comprehension 

Understand 

Application 

Apply 

Analysis 

Analyze 

Synthesis 

Evalnate 

Evaluate 

Create 

A.  Mastery  of  knowledge, 
techniques,  skills,  modern  tools 
of  disciplines. 

Factual  Knowledge 

Conceptual 

Knowledge 

Procedural 

Knowledge 

Meta-Cognitive 

Knowledge 

B.  Apply  current  knowledge 
and  adapt  to  emerging 
applications  of  math,  science, 
engineering,  and  technology. 

NIU  Gen  Ed  Goals  - Students: 
a.  Hi.  perform  basic 
computations,  display 
facility  with  use  of  formal 
and  quantitative  reasoning 
analysis  and  problem 
solving,  and  interpret 
mathematical  models  and 
statistical  information. 

Factual  Knowledge 

Conceptual 

Knowledge 

Procedural 

Knowledge 

Meta-Cognitive 

Knowledge 
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ABET/TAC/NAIT  Engineering 
& Technology  Outcomes 

Bloom's 

Knowledge 

Dimension 

Dale’s  Cone 

Student  Learning 
Objectives 

Bloom’s  Cognitive  Process  Dimension 

Passive 

Participating 

Active 

Knowledge 

Remember 

Comprehension 

Understand 

Application 

Apply 

Analysis 

Analyze 

Synthesis 

Evalnate 

Evaluate 

Create 

C.  Conduct,  analyzes,  and 
interprets  experiments;  apply 
experimental  results  to  improve 
processes. 

Gen  Ed  Goals  - Students: 
b.  develop  an  ability  to  use 
modes  of  inquiry  across  a 
variety  of  disciplines  in  the 
humanities  and  the  arts,  the 
physical  sciences  and 
mathematics,  and  social 
sciences. 

b.iv.  demonstrate  an  ability 
to  use  scientific  methods  and 
theories  to  understand  the 
phenomena  studied  in  the 
natural  and  social  sciences. 

Factual  Knowledge 

Conceptual 

Knowledge 

Procedural 

Knowledge 

Meta-Cognitive 

Knowledge 

D.  Ability  to  apply  creativity  in 
the  design  of  systems, 
components,  or  processes 
appropriate  to  program 
objectives. 

NIU  Gen  Ed  Goals  - 
Students: 

c.  develop  an  understanding 
of  the  relatedness  of  various 
disciplines  by  integrating 
knowledge  from  several 
disciplines  and  applying  that 
knowledge  to  an 
understanding  of  important 
problems  and  issues. 

Factual  Knowledge 

Conceptual 

Knowledge 

Procedural 

Knowledge 

Meta-Cognitive 

Knowledge 
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ABET/TAC/NAIT  Engineering 
& Technology  Outcomes 

Bloom's 

Knowledge 

Dimension 

Dale’s  Cone 

Student  Learning 
Objectives 

Bloom’s  Cognitive  Process  Dimension 

Passive 

Participating 

Active 

Knowledge 

Remember 

Comprehension 

Understand 

Application 

Apply 

Analysis 

Analyze 

Synthesis 

Evalnate 

Evaluate 

Create 

E. 

tei 

Function  effectively  on 
ams. 

NIU  Gen  Ed  Goals- 
Students: 
b.iii.  demonstrate  a 
knowledge  of  cultural 
traditions  and  philosophical 
ideas  that  have  shaped 
societies,  civilizations,  and 
human  self-conceptions, 
d.  develop  social 
responsibility  and 
preparation  for  citizenship 
through  global  awareness, 
environmental  sensitivity, 
and  an  appreciation  of 
cultural  diversity. 

Factual  Knowledge 

Conceptual 

Knowledge 

Procedural 

Knowledge 

Meta-Cognitive 

Knowledge 

F.  Identify,  analyze,  and  solve 
technical  problems. 

Gen  Ed  Goals  - Students: 
b.  develop  an  ability  to  use 
modes  of  inquiry  across  a 
variety  of  disciplines  in  the 
humanities  and  the  arts,  the 
physical  sciences  and 
mathematics,  and  social 
sciences. 

b.iv.  demonstrate  an  ability 
to  use  scientific  methods  and 
theories  to  understand  the 
phenomena  studied  in  the 
natural  and  social  sciences. 

Factual  Knowledge 

Conceptual 

Knowledge 

Procedural 

Knowledge 

Meta-Cognitive 

Knowledge 
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ABET/TAC/NAIT  Engineering 
& Technology  Outcomes 

Bloom's 

Knowledge 

Dimension 

Dale’s  Cone 

Student  Learning 
Objectives 

Bloom’s  Cognitive  Process  Dimension 

Passive 

Participating 

Active 

Knowledge 

Remember 

Comprehension 

Understand 

Application 

Apply 

Analysis 

Analyze 

Synthesis 

Evalnate 

Evaluate 

Create 

G.  Communicate  effectively  in 
writing. 

NIU  Gen  Ed  Goals- 
Students: 

a.  develop  habits  of  writing, 
speaking,  and  reasoning 
necessary  for  continued 
learning. 

a.i.  communicate  clearly  in 
written 

English,  demonstrating 
ability  to 

comprehend,  analyze,  and 

Factual  Knowledge 

Conceptual 

Knowledge 

Procedural 

Knowledge 

Meta-Cognitive 

Knowledge 

H.  Communicate  effectively 
orally. 

Factual  Knowledge 

Conceptual 

Knowledge 

Procedural 

Knowledge 

Meta-Cognitive 

Knowledge 
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ABET/TAC/NAIT  Engineering 
& Technology  Outcomes 

Bloom's 

Knowledge 

Dimension 

Dale’s  Cone 

Student  Learning 
Objectives 

Bloom’s  Cognitive  Process  Dimension 

Passive 

Participating 

Active 

Knowledge 

Remember 

Comprehension 

Understand 

Application 

Apply 

Analysis 

Analyze 

Synthesis 

Evalnate 

Evaluate 

Create 

NIU  Gen  Ed  Goals  - 
Students: 

a.  develop  habits  of  writing, 
speaking,  and  reasoning 
necessary  for  continued 
learning. 

a.i.  communicate  clearly  in 
written  English, 
demonstrating  ability  to 
comprehend,  analyze,  and 
interrogate  critically, 
aii.  communicate  in  a manner 
that  unites  theory,  criticism, 
and  practice  in  speaking  & 
writing. 

I.  Recognize  the  need  for,  and 
an  ability  to  engage  in  life  long 
learning. 

NIU  Gen  Ed  Goals  - 
Students: 

a.iv.  are  able  to  access  and 
use  various  information 
sources. 

Internet,  text,  and  field  case. 

Factual  Knowledge 

Conceptual 

Knowledge 

Procedural 

Knowledge 

Meta-Cognitive 

Knowledge 
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ABET/TAC/NAIT  Engineering 
& Technology  Outcomes 

Bloom's 

Knowledge 

Dimension 

Dale’s  Cone 

Student  Uearning 
Objectives 

Bloom’s  Cognitive  Process  Dimension 

Passive 

Participating 

Active 

Knowledge 

Remember 

Comprehension 

Understand 

Application 

Apply 

Analysis 

Analyze 

Synthesis 

Evalnate 

Evaluate 

Create 

J.  Understand  professional, 
ethical,  and  social 
responsibilities. 

NIU  Gen  Ed  Goals  - 
Students: 
d.  develop  social 
responsibility  and 
preparation  for  citizenship 
through  global  awareness, 
environmental  sensitivity, 
and  an  appreciation  of 
cultural  diversity. 

Factual  Knowledge 

Conceptual 

Knowledge 

Procedural 

Knowledge 

Meta-Cognitive 

Knowledge 

K.  Respect  for  diversity  and  a 
knowledge  of  contemporary 
professional,  societal,  and  globa 
issues. 

NIU  Gen  Ed  Goals  - 
Students: 

d.  develop  social  respons- 
ibility and  preparation  for 
citizenship  through  global 
awareness,  environmental 
sensitivity,  and  an  appre- 
ciation of  cultural  diversity. 

1 

Factual  Knowledge 

Conceptual 

Knowledge 

Procedural 

Knowledge 

Meta-Cognitive 

Knowledge 

E.  Commitment  to  quality, 
timeliness,  and  continuous 
improvement. 

Factual  Knowledge 

Conceptual 

Knowledge 

Procedural 

Knowledge 

Meta-Cognitive 

Knowledge 
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ABET/TAC/NAIT  Engineering 
& Technology  Outcomes 

Bloom's 

Knowledge 

Dimension 

Dale’s  Cone 

Student  Learning 
Objectives 

Bloom’s  Cognitive  Process  Dimension 

Passive 

Participating 

Active 

Knowledge 

Remember 

Comprehension 

Understand 

Application 

Apply 

Analysis 

Analyze 

Synthesis 

Evalnate 

Evaluate 

Create 

M.  Ability  to  program 
computers  and/or  utilize 
computer  applications 
effectively. 

Factual  Knowledge 

Conceptual 

Knowledge 

Procedural 

Knowledge 

Meta-Cognitive 

Knowledge 

N.  Ability  to  use  modern 
laboratory  techniques,  skills, 
and/or  equipment  effectively. 

Factual  Knowledge 

Conceptual 

Knowledge 

Procedural 

Knowledge 

Meta-Cognitive 

Knowledge 

O.  Ability  to  manage  projects 
effectively. 

NIU  Gen  Ed  Goals  - 
Students: 

c.  develop  an  understanding  of 
the  interrelatedness  of  various 
disciplines  by  integrating 
knowledge  from  several 
disciplines  and  applying  that 
knowledge  to  an  understanding 
of  important  problems  and 
issues. 

Factual  Knowledge 

Conceptual 

Knowledge 

Procedural 

Knowledge 

Meta-Cognitive 

Knowledge 
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ABET/TAC/NAIT  Engineering 
& Technology  Outcomes 

Bloom's 

Knowledge 

Dimension 

Dale’s  Cone 

Student  Learning 
Objectives 

Bloom’s  Cognitive  Process  Dimension 

Passive 

Participating 

Active 

Knowledge 

Remember 

Comprehension 

Understand 

Application 

Apply 

Analysis 

Analyze 

Synthesis 

Evalnate 

Evaluate 

Create 

P.  Ability  to  design,  manipulate, 
and  manage  industrial  systems. 

Factual  Knowledge 

Conceptual 

Knowledge 

Procedural 

Knowledge 

Meta-Cognitive 

Knowledge 

NIU  Gen  Ed  Goals  - 
Students: 

c.  develop  an  understanding 
of  the  interrelatedness  of 
various  disciplines  by 
integrating  knowledge  from 
several  disciplines  and 
applying  that  knowledge  to 
an  understanding  of 
important  problems  and 
issues. 

Q.  Ability  to  manage  or  lead 
personnel  effectively. 

Factual  Knowledge 

Conceptual 

Knowledge 

Procedural 

Knowledge 

Meta-Cognitive 

Knowledge 

Addition;  Additional  educational  outcomes  -NIU  General  Education  Goals  - articulated  by  the  overall  program 
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ABET  Engineering  Outcomes 


Bloom's 

Knowledge 

Dimension/ 


Dale’s  Cone 
Passive/Active 


Student  Learning  Objectives 


Bloom’s  Cognitive  Process  Dimension 


Knowledge 

Remember 


Comprehension 

Understand 


Application  Analysis 
Apply  Analyze 


Synthesis 

Evaluate 


Evaluate 

Create 


A.  Apply  knowledge  of  math, 
science,  engineering 


NIU  Gen  Ed  Goals  - 
Students: 

a.  Hi.  perform  basic 
computations,  display 
facility  with  use  of 
formal  and  quantitative 
reasoning  analysis  and 
problem  solving,  and 
interpret  mathematical 
models  and  statistical 
information. 


Factual 

Knowledge 

Conceptual 

Knowledge 

Procedural 

Knowledge 

Meta- 

Cognitive 

Knowledge 


I.  To  learn  to  use  a specific  set  of  analytical 
tools  for  technical  decision  making  under 
uncertainty. 

A.  Students  will  be  able  to 

construct/create  a decision  tree  to  aid 
in  determining  the  best  course  of 
action  for  a given  set  of 
circumstances 

1 . To  define  the  states  of  nature  of 
the  system,  process,  or  situation 

2.  To  develop  the  branch  structure 
of  the  tree 

a.  To  identify  decision  nodes; 
what  are  the  items  the 
decision  maker  chooses 

b.  To  identify  the  chance 
nodes;  the  events  that 
occur  by  chance  with  a 
given  probability 

c.  To  draw  the  arcs,  which 
define  the  sequences  and 
relationships  between 
nodes 

3.  To  identify  the  outcomes 

a.  To  define  the  choices  for  a 
decision  node 

b.  To  define  the  possible 
outcomes  of  a chance 
node,  which  are  a set  of 
mutually  exclusive 
outcomes 

c.  To  define  the 
“consequence,”  or  the  final 
outcome  of  a branch 

4.  To  solve  for  the  expected  value 
of  the  decision  tree  (EV,  EMV) 
a.  To  construct  the  joint, 

conditional,  and  marginal 


☆- 


☆ 

☆ 


☆ 

☆ 


☆ 
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ABET  Engineering  Outcomes 

Bloom's 

Knowledge 

Dimension/ 

Dale’s  Cone 
Passive/Active 

Student  Learning  Objectives 

Bloom’s  Cognitive  Process  Dimension 

Knowledge 

Remember 

Comprehension 

Understand 

Application 

Apply 

Analysis 

Analyze 

Synthesis 

Evaluate 

Evaluate 

Create 

probabilities 

b.  To  calculate  all  branch 
probabilities  of  the  tree 

i.  To  apply  Baye’s 
Theorem 

ii. To  implement  the 
inverse  tree  structure 
technique 

5.  To  find  and  compare  the 

expected  value  of  both  sample 
and  perfect  information  (EVPI, 
EVSI) 

a.  To  construct  the  decision 

W 

w 

trees  to  calculate  EVPI  and 
EVSI 

b.  To  evaluate  the  relevance 
and  importance  of  the 
values  obtained  for  EVPI 
and  EVSI  to  the  decision 
process 

B.  Students  will  be  able  to  construct 
the  formulae  for  conditional 
likelihood  ratios  and  to  calculate  the 
probabilities/ratios 

1 . To  calculate  the  conditional 
likelihood  ratio  (CLR) 
associated  with  a particular 
observation 

2.  To  calculate  the  CLR  for 
multiple  observations 

3.  To  compare  these  results  with 

W 

Baye’s  Theorem 

C.  Students  will  be  able  to  construct  a 
single-attribute  utility  function 

1 . To  propose  a lottery  that  would 
be  appropriate  for  evaluation  of 
risk 

2.  To  calculate  the  necessary 
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ABET  Engineering  Outcomes 

Bloom's 

Knowledge 

Dimension/ 

Dale’s  Cone 
Passive/Active 

Student  Learning  Objectives 

Bloom’s  Cognitive  Process  Dimension 

Knowledge 

Remember 

Comprehension 

Understand 

Application 

Apply 

Analysis 

Analyze 

Synthesis 

Evaluate 

Evaluate 

Create 

values  from  the  lotteries  needed 

for  analysis 

a.  To  calculate  the  certainty 
equivalent 

b.  To  calculate  the  risk 
premium 

c.  To  calculate  the  selling 
price 

d.  To  calculate  the  buying 
price 

e.  To  calculate  the  insurance 
premium 

3.  To  translate  the  lotteries  into  a 

mathematical  function 
4.  To  create  a graphical 

interpretation  of  the  function 
5.  To  analyze  and  compare  two 
lotteries  at  a time,  to  be  used 
when  a reference  point  is 
needed 

D.  Students  will  be  able  to  develop  and 
analyze  fault  trees 
1 . To  describe  the  events  of  a tree 
for  a given  scenario 

PA 

PA 

a.  To  identify  the  top  event 

b.  To  define  primary  and 
secondary  failures  and 
command  faults 

c.  To  identify  the  sequence  of 
events 

2.  To  create  the  fault  tree  for  a 
given  scenario  (such  as  the 
safety  analysis  of  a system) 
using  deductive  analysis 

a.  To  define  the  “and”  and 
“or”  gates 

b.  To  implement  the  logic 
symbols  into  the  tree 

h\ 
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ABET  Engineering  Outcomes 

Bloom's 

Knowledge 

Dimension/ 

Dale’s  Cone 
Passive/Active 

Student  Learning  Objectives 

Bloom’s  Cognitive  Process  Dimension 

Knowledge 

Remember 

Comprehension 

Understand 

Application 

Apply 

Analysis 

Analyze 

Synthesis 

Evaluate 

Evaluate 

Create 

3.  To  develop  and  analyze  dual 
fault  trees 

a.  To  translate  the  meaning  of 
a system  failure  into  the 
reliability  of  the  system 
E.  Students  will  be  able  to  construct  a 
multi-attribute  utility  function 

1 . To  construct  a graphical  model 
for  the  function 

2.  To  develop  a mathematical 
model  for  the  function 

☆ 

☆ 

B. 

an 

Design  and  conduct  experimen 
lalyze  and  interpret  data 

NIU  Gen  Ed  Goals  - Students: 
a.iii.  perform  basic 
computations,  display 
facility  with  use  of  formal 
and  quantitative  reasoning 
analysis  and  problem 
solving,  and  interpret 
mathematical  models  and 
statistical  information. 

ts; 

N/A 
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ABET  Engineering  Outcomes 

Bloom's 

Knowledge 

Dimension/ 

Dale’s  Cone 
Passive/Active 

Student  Learning  Objectives 

Bloom’s  Cognitive  Process  Dimension 

Knowledge 

Remember 

Comprehension 

Understand 

Application 

Apply 

Analysis 

Analyze 

Synthesis 

Evaluate 

Evaluate 

Create 

C.  Design  a system,  component, 

Factual 

A 

III.  To  structure  and  solve  complicated 

w 

process  to  meet  desired  needs  within 
realistic  constraints,  e.g.  economic, 
environmental,  social,  political, 
ethical,  health,  safety, 
mannfacturability,  & snstainability. 

Knowledge 

Conceptual 

Knowledge 

Procedural 

decision  problems 

B.  Students  will  be  able  to  define  a 
scenario  to  be  analyzed  with  the 
decision  analysis  techniques 
1 . To  research  an  area  in  which  the 
tools  can  be  applied,  and  to 

W 

NIU  Gen  Ed  Goals  - 
Students: 

c.  develop  an  understanding 
of  the  relatedness  of  various 
disciplines  by  integrating 
knowledge  from  several 
disciplines  and  applying  that 
knowledge  to  an 
understanding  of  important 
problems  and  issues. 

Knowledge 

Meta- 

Cognitive 

Knowledge 

choose  a problem  for  study 

2.  To  define  the  scope  for  the 
problem  chosen 

3.  To  determine  a set  of  objectives 
for  the  given  problem 

C.  Students  will  be  able  to 

formulate/design  a possible  solution 
approach  to  the  given  problem 
1.  To  determine  an  appropriate 
solution  technique  to  be  applied 
to  the  problem  from  the  set  of 

W 

p^ 

available  tools 

2.  To  formulate  the  solution 

D.  Students  will  be  able  to  implement 
the  solution  techniques  to  obtain  a 
first  round  solution 

1 . To  solve  the  formulation 

E.  Students  will  be  able  to  analyze  the 
solution  found  in  order  to  assess  the 
current  state 

1.  To  assess/analyze  the  value  of 
the  solution  found  (does  it  make 
sense  numerically?) 

2.  To  evaluate  the  significance  of 
the  solution  (what  does  this 
mean  for  the  decision  maker?) 

F.  Students  will  be  able  to  recommend 

w 

a course  of  action  based  on  the 
original  solution 
1 . To  establish  if  the  decision 
maker  is  risk  averse,  risk  prone. 
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ABET  Engineering  Outcomes 

Bloom's 

Knowledge 

Dimension/ 

Dale’s  Cone 
Passive/Active 

Student  Learning  Objectives 

Bloom’s  Cognitive  Process  Dimension 

Knowledge 

Remember 

Comprehension 

Understand 

Application 

Apply 

Analysis 

Analyze 

Synthesis 

Evaluate 

Evaluate 

Create 

or  risk  neutral 

2.  To  recommend  the  path  that  the 
decision  maker  should  embark 
upon,  based  on  the  results 
G.  Students  will  be  able  to  assess  the 
usefulness  of  the  solution  and 

recommend  any  changes  in  the 
process 

1 . To  evaluate  if  the  current 
solution  adequately  answers  the 
most  important  questions  facing 
the  decision  maker 

2.  To  analyze  the  solution  for 
computational  accuracy 

3.  To  decide  if  the  solution  should 
be  improved  upon 

a.  To  conclude  if  factors 
(attributes)  need  to  be 
added 

b.  To  conclude  if  factors 
(attributes)  need  to  be 
removed 

c.  To  determine  if  the 
correct  probabilities 
were  utilized 
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ABET  Engineering  Outcomes 

Bloom's 

Knowledge 

Dimension/ 

Dale’s  Cone 
Passive/Active 

Student  Learning  Objectives 

Bloom’s  Cognitive  Process  Dimension 

Knowledge 

Remember 

Comprehension 

Understand 

Application 

Apply 

Analysis 

Analyze 

Synthesis 

Evaluate 

Evaluate 

Create 

D.  Function  on  multi-disciplinary 

Factual 

Knowledge 

Conceptual 

Knowledge 

Procedural 

Knowledge 

Meta- 

Cognitive 

Knowledge 

A 

IV.  To  demonstrate  an  ability  to 

effectively  present  the  problem, 
solution,  and  recommendations  of  a 
complicated  decision  scenario 

A.  Students  will  explain  results  via  a 
formal  presentation 

B.  Students  will  explain  results  via  a 
formal  written  technical  report 

w 

teams. 

NIU  Gen  Ed  Goals- 
Students: 
b.iii.  demonstrate  a 
knowledge  of  cultural 
traditions  and  philosophical 
ideas  that  have  shaped 
societies,  civilizations,  and 
human  self-conceptions, 
d.  develop  social 
responsibility  and 
preparation  for  citizenship 
through  global  awareness, 
environmental  sensitivity, 
and  an  appreciation  of 
cultural  diversity. 

PA 
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ABET  Engineering  Outcomes 

Bloom's 

Knowledge 

Dimension/ 

Dale’s  Cone 
Passive/Active 

Student  Learning  Objectives 

Bloom’s  Cognitive  Process  Dimension 

Knowledge 

Remember 

Comprehension 

Understand 

Application 

Apply 

Analysis 

Analyze 

Synthesis 

Evaluate 

Evaluate 

Create 

E.  Identify,  formulate,  and  solve 

Factual 

A 

II.  To  apply  analytic  decision  making 

engineering,  problems 

Knowledge 

Conceptual 

Knowledge 

techniques  for  technical  decision  making 
under  uncertainty,  and  to  analyze  and 
evaluate  the  results. 

A.  Students  will  be  able  to  perform 
sensitivity  analysis  on  a decision 

h\ 

Gen  Ed  Goals  - Students; 
b.  develop  an  ability  to  use 

modes  of  inquiry  across  a 
variety  of  disciplines  in  the 
humanities  and  the  arts,  the 
physical  sciences  and 
mathematics,  and  social 
sciences. 

b.iv.  demonstrate  an  ability 
to  use  scientific  methods  and 
theories  to  understand  the 
phenomena  studied  in  the 
natural  and  social  sciences. 

Procedural 

Knowledge 

tree  scenario 

I . To  construct  a graph  to 

JW 

Meta- 

Cognitive 

Knowledge 

interpret  the  results 

a.  To  plot  the  two 

dimensional  plane  for  a 
single  chance  node 
sequence 

i.  To  interpret  the  graph 

b.  To  plot  the  plane  for  two 
sequential  chance  events 
(plot  the  pq  plane) 

i.  To  interpret  the  graph 
2.  To  determine  the  threshold 
probability  levels 

a.  To  decide  if  the  current 
solution  is  a good  one 
and  justify  the  decision 

b.  To  determine  the  salient 
factors  (variables)  in  the 
decision  being  modeled 
by  the  current  tree 

c.  To  make 
recommendations  for 
improvements 

H.  Students  will  be  able  to  make 

inferences  about  a system  based  on 
the  values  of  the  conditional 
likelihood  ratios 

1 . To  determine  the  pass  or  fail 
rates  allowed  for  a given  set  of 
specification  limit 

2.  To  determine  probabilities  of 
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ABET  Engineering  Outcomes 

Bloom's 

Knowledge 

Dimension/ 

Dale’s  Cone 
Passive/Active 

Student  Learning  Objectives 

Bloom’s  Cognitive  Process  Dimension 

Knowledge 

Remember 

Comprehension 

Understand 

Application 

Apply 

Analysis 

Analyze 

Synthesis 

Evaluate 

Evaluate 

Create 

occurrence  for  multi-variate 
systems  based  on  the  values  of 
the  CLR’ s 

I.  Students  will  be  able  to  evaluate  a 

given  utility  function 

1 . To  analyze  the  degree  of  risk 
aversion  from  the  utility 
function;  risk  prone,  risk 
averse,  risk  neutral 

2.  To  assess  the  process  being 
modeled;  is  the  model 
sufficient 

a.  To  determine  if  the 
model  needs  to  have 
more  attributes 

3.  To  recommend  any  changes  in 

the  model 

a.  To  decide  if  more 
iterations  are  necessary 
for  the  lotteries 

b.  To  decide  if  any  of  the 
lotteries  need  to  be 
referenced 

c.  To  determine  if  the  utility 
function  is  consistent 
with  the  behavior  of  the 
decision  maker 

J.  Students  will  be  able  to  evaluate  the 

scenarios  modeled  by  a fault  tree 

1 . To  determine  the  reliability  and 
failure  probability  relationships 

2.  To  propagate  the  probabilities 
through  the  gates 

3.  To  perform  a qualitative 
evaluation 

a.  To  determine  cut  sets,  as 
well  as  the  minimal  cut 
set 

h\ 
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ABET  Engineering  Outcomes 

Bloom's 

Knowledge 

Dimension/ 

Dale’s  Cone 
Passive/Active 

Student  Learning  Objectives 

Bloom’s  Cognitive  Process  Dimension 

Knowledge 

Remember 

Comprehension 

Understand 

Application 

Apply 

Analysis 

Analyze 

Synthesis 

Evaluate 

Evaluate 

Create 

4.  To  perform  a quantitative 
evaluation 

a.  To  determine  cut  sets,  as 
well  as  the  minimal  cut 
set 

b.  To  develop  the 
equivalent  fault  tree 

c.  To  obtain  the  numerical 
probability  that  a given 
cut  set  induces  failure  of 
the  system 

5.  To  implement  the  additive 
model  for  multi-attribute 
scenarios 

K.  Students  will  be  able  to  implement 

the  additive  model  for  multi- 
attribute utility  theory 

1.  To  assess  a two-attribute  utility 
function 

2.  To  determine  the  weights  of  the 
functions  (the  k’ s) 
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ABET  Engineering  Outcomes 

Bloom's 

Knowledge 

Dimension/ 

Dale’s  Cone 
Passive/Active 

Student  Learning  Objectives 

Bloom’s  Cognitive  Process  Dimension 

Knowledge 

Remember 

Comprehension 

Understand 

AppUcation 

Apply 

Analysis 

Analyze 

Synthesis 

Evaluate 

Evaluate 

Create 

F.  Understand  professional  and 
ethical  responsibility 

NIU  Gen  Ed  Goals  - Students: 
d.  develop  social  responsibility  and 
preparation  for  citizenship  through 
global  awareness,  environmental 
sensitivity,  and  an  appreciation  of 
cultural  diversity. 

N/A 

G.  Communicate  effectively 

Factual 

Knowledge 

Conceptual 

Knowledge 

Procedural 

Knowledge 

Meta- 

Cognitive 

Knowledge 

A 

V.  To  demonstrate  an  ability  to  effectively 
present  the  problem,  solution,  and 
recommendations  of  a complicated 
decision  scenario 

A.  Students  will  explain  results  via  a 
formal  presentation 

B.  Students  will  explain  results  via  a 
formal  written  technical  report 

NIU  Gen  Ed  Goals-  Students: 
a.  develop  habits  of  writing, 
speaking,  and  reasoning  necessary 
for  continued  learning, 
a.i.  communicate  clearly  in  written 
English,  demonstrating  ability  to 
comprehend,  analyze,  and 
interrogate  critically. 
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ABET  Engineering  Outcomes 

Bloom's 

Knowledge 

Dimension/ 

Dale’s  Cone 
Passive/Active 

Student  Learning  Objectives 

Bloom’s  Cognitive  Process  Dimension 

Knowledge 

Remember 

Comprehension 

Understand 

Application 

Apply 

Analysis 

Analyze 

Synthesis 

Evaluate 

Evaluate 

Create 

H.  understand  impact  of  engineering 
solutions  in  a global  economic, 
environmental,  societal  context 

NIU  Gen  Ed  Goals  - Students: 
d.  develop  social  responsibility  and 
preparation  for  citizenship  through 
global  awareness,  environmental 
sensitivity,  and  an  appreciation  of 
cultural  diversity. 

N/A 

I.  Recognize  the  need  for,  and  have 

Factual 

Knowledge 

Conceptual 

Knowledge 

Procedural 

Knowledge 

Meta- 

Cognitive 

Knowledge 

A 

V.  To  demonstrate  an  ability  to  effectively 
present  the  problem,  solution,  and 
recommendations  of  a complicated 
decision  scenario 

w 

capaDiiiiy  lu  engage  in  me  lung 
learning. 

NIU  Gen  Ed  Goals  - Students: 
a.iv.  are  able  to  access  and  use 
various  information  sources. 
Internet,  text,  and  field  case. 

J.  knowledge  of  contemporary  issues 

NIU  Gen  Ed  Goals  - Students: 
d.  develop  social  responsibility  and 
preparation  for  citizenship  through 
global  awareness,  environmental 
sensitivity,  and  an  appreciation  of 
cultural  diversity. 

Factual 

Knowledge 

Conceptual 

Knowledge 

Procedural 

Knowledge 

Meta- 

Cognitive 

Knowledge 

A 

IV.  To  identify  and  define  any  limitations  of 
the  models  and  techniques  for  rational 
decision-making. 
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ABET  Engineering  Outcomes 

Bloom's 

Knowledge 

Dimension/ 

Dale’s  Cone 
Passive/Active 

Student  Learning  Objectives 

Bloom’s  Cognitive  Process  Dimension 

Knowledge 

Remember 

Comprehension 

Understand 

Application 

Apply 

Analysis 

Analyze 

Synthesis 

Evaluate 

Evaluate 

Create 

K.  use  techniques,  skills,  and  modern 
engineering  tools  necessary  for 
engineering  practice 

Factual 

Knowledge 

Conceptual 

Knowledge 

Procedural 

Knowledge 

Meta- 

Cognitive 

Knowledge 

A 

IV.  To  identify  and  define  any  limitations  of 
the  models  and  techniques  for  rational 
decision-making. 

V.  To  demonstrate  an  ability  to  effectively 
present  the  problem,  solution,  and 
recommendations  of  a complicated 
decision  scenario 

w 

Addition:  Additional  educational  outcomes  -NIU  General  Education  Goals  - articulated  by  the  overall  program 
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Table  B.7.b.2:  AbulAzad  May  15‘''  2006 


ABET/TAC/NAIT 
Engineering  & Technology 
Outcomes 

Bloom's 

Knowledge 

Dimension 

Dale’s  Cone 

Student  Learning  Objectives 

Bloom’s  Cognitive  Process  Dimension 

Passive 

Participating 

Active 

Knowledge 

Remember 

Comprehension 

Understand 

AppUcation 

Apply 

Analysis 

Analyze 

Synthesis 

Evaluate 

Evaluate 

Create 

A.  Mastery  of  knowledge, 
techniques,  skills,  modern  tools  of 
disciplines. 

J Factual 
Knowledge 

J Conceptual 
Knowledge 

J Procedural 
Knowledge 

J Meta-Cognitive 
Knowledge 

Passive 

Active 

and 

passive 

1.  To  examine  the  components  of  a digital 
system. 

a.  To  contrast  between  analog  and  digital  signals 

b.  To  classify  binary  digits,  logic  levels,  and 
digital  waveforms 

c.  To  compare  basic  logic  operations 

d.  To  categorize  fixed  function  integrated  circuits 

e.  To  interpret  the  operation  of  simple  digital 
systems 

2.  To  examine  the  structures  for  various 
number  systems. 

a.  To  distinguish  between  various  parts  of  number 
systems. 

b.  To  examine  the  counting  in  binary,  octal, 
decimal,  and  octal. 

1 

Active 

and 

passive 

3.  To  distinguish  the  conversion  methods  for 
various  number  systems. 

a.  To  convert  between  binary  and  decimal 

b.  To  convert  between  binary  and  hexadecimal 

c.  To  convert  between  binary  and  octal 

Passive 

4.  To  perform  different  binary  arithmetic 
operations:  addition,  subtraction,  I's 
complement,  2's  complement,  and  signed 
numbers. 

a.  To  examine  the  basic  rules  involving  each  of  the 
operations. 

b.  To  use  the  rules  to  perform  each  of  the 
operations. 

Active 

and 

passive 

5.  To  examine  the  operation  and  use  of  various 
logic  gates  with  different  input  patterns:  AND, 
OR,  and  NOT,  NAND,  NOR,  XOR  and  XNOR. 

a.  To  develop  the  truth  tables  of  various  logic 
gates  using  established  rules. 
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Dale’s  Cone 

Student  Learning  Objectives 
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Passive 

Participating 

Active 

Knowledge 

Remember 

Comprehension 

Understand 

AppUcation 

Apply 

Analysis 

Analyze 

Synthesis 

Evaluate 

Evaluate 

Create 

b.  To  use  the  truth  tables  to  identify  output  pattern 
of  a logic  gate  for  a given  set  of  input. 

c.  To  predict  output  logic  levels  for  a pulse  input 
pattern. 

d.  To  recommend  the  use  of  appropriate  logic 
gate(s)  for  a given  application. 

Active 

and 

passive 

6.  To  analyze  the  properties  of  fixed-function 
logic  integrated  circuits  (IC):  Complementary 
Metal  Oxide  Semiconductor  (CMOS)  and 
Transistor-Transistor  Logic  (TTL). 

a.  To  identify  various  supply  voltage  and  power 
requirements  for  CMOS  and  TTL  ICs. 

b.  To  analyze  the  generic  numbering  convention 
for  CMOS  and  TTL  ICs. 

c.  To  classify  common  logic  gate  ICs  according  to 
their  standard  identifier  digit. 

d.  To  examine  the  logic  gate  configuration  within 
an  IC. 

e.  To  compare  alternative  logic  symbols  for 
representing  logic  gates  while  drawing  a circuit 
diagram. 

f.  To  examine  the  voltage  values  for  input  output 
logic  levels  for  CMOS  and  TTL  ICs. 

1 

Active 

and 

passive 

8.  To  use  Laws  and  Rules  of  Boolean  algebra 
and  DeMorgan’s  Theorems  for  manipulating 
Boolean  expressions. 

a.  To  use  the  commutative,  associative,  and 
distributive  laws  to  manipulate  Boolean 
expressions. 

b.  To  examine  the  use  of  Boolean  rules  while 
manipulating  Boolean  expressions. 

c.  To  use  DeMorgan’s  Theorems  for  manipulating 
Boolean  expressions. 

d.  To  adapt  the  Boolean  laws.  Boolean  rules,  and 
DeMorgan’ s Theorems  while  minimizing  Boolean 
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Bloom’s  Cognitive  Process  Dimension 

Passive 

Participating 

Active 

Knowledge 

Remember 

Comprehension 

Understand 

Application 

Apply 

Analysis 

Analyze 

Synthesis 

Evaluate 

Evaluate 

Create 

Active 

and 

passive 

Active 

and 

passive 

Active 

expressions. 

9.  To  analyze  digital  logic  circuits  using  Boolean 
algebra. 

a.  To  develop  a Boolean  expression  for  a given 
logic  circuit. 

b.  To  evaluate  a Boolean  expression  and  prepare  a 
truth-table  for  the  logic  circuit. 

c.  To  demonstrate  the  use  of  Boolean  algebra 
while  minimizing  Boolean  expressions. 

12.  To  minimize  logic  expressions  using 
Karnaugh  map  (K-map). 

a.  To  develop  K-maps  with  different  size  of  input 
variables  (1  to  4). 

b.  To  map  SOP  expressions  on  K-maps. 

c.  To  develop  minimized  expressions  from  K- 
maps.. 

d.  To  construct  K-map  from  a non-structured  SOP 
expression. 

15.  To  evaluate  the  properties  of  Latches,  Flip- 
Flops,  and  timers. 

a.  To  contrast  between  Latches  and  Flip-Flops. 

b.  To  evaluate  the  properties  of  edge-triggered  J-K 
Flip-Flop. 

c.  To  evaluate  the  properties  of  edge-triggered  D 
Flip-Flop. 

d.  To  evaluate  the  properties  of  edge-triggered  S-R 
Flip-Flop. 

e.  To  utilize  the  asynchronous  Preset  and  Clear 
inputs  of  Flip-Flops. 

f.  To  examine  the  operating  characteristics  of  Flip- 
flops,  such  as-  propagation  delay  times,  set-up 
time,  hold  time.  Maximum  clock  frequency.  Pulse 
width,  and  Power  dissipation. 

g.  To  compare  the  properties  of  commercially 
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Passive 

Participating 

Active 

Knowledge 

Remember 

Comprehension 

Understand 

AppUcation 

Apply 

Analysis 

Analyze 

Synthesis 

Evaluate 

Evaluate 

Create 

available  Flip-Flop  ICs. 

B.  Apply  current  knowledge  and 
adapt  to  emerging  applications  of 
math,  science,  engineering,  and 
technology. 

J Factual 
Knowledge 

J Conceptual 
Knowledge 

Active 

and 

Passive 

3.  To  distinguish  the  conversion  methods  for 
various  number  systems. 

a.  To  convert  between  binary  and  decimal 

b.  To  convert  between  binary  and  hexadecimal 

c.  To  convert  between  binary  and  octal 

NIU  Gen  Ed  Goals  - Students: 
a.  Hi.  perform  basic 
computations,  display 
facility  with  use  of  formal 
and  quantitative  reasoning 
analysis  and  problem 
solving,  and  interpret 
mathematical  models  and 
statistical  information. 

J Procedural 
Knowledge 

J Meta-Cognitive 
Knowledge 

Active 

and 

passive 

6.  To  analyze  the  properties  of  fixed-function 
logic  integrated  circuits  (IC):  Complementary 
Metal  Oxide  Semiconductor  (CMOS)  and 
Transistor-Transistor  Logic  (TTL). 

a.  To  identify  various  supply  voltage  and  power 
requirements  for  CMOS  and  TTL  ICs. 

b.  To  analyze  the  generic  numbering  convention 
for  CMOS  and  TTL  ICs. 

c.  To  classify  common  logic  gate  ICs  according  to 
their  standard  identifier  digit. 

d.  To  examine  the  logic  gate  configuration  within 
an  IC. 

e.  To  compare  alternative  logic  symbols  for 
representing  logic  gates  while  drawing  a circuit 
diagram. 

f.  To  examine  the  voltage  values  for  input  output 
logic  levels  for  CMOS  and  TTL  ICs. 

Active 

and 

passive 

9.  To  analyze  digital  logic  circuits  using  Boolean 
algebra. 

a.  To  develop  a Boolean  expression  for  a given 
logic  circuit. 

b.  To  evaluate  a Boolean  expression  and  prepare  a 
truth-table  for  the  logic  circuit. 

c.  To  demonstrate  the  use  of  Boolean  algebra 
while  minimizing  Boolean  expressions. 

12.  To  minimize  logic  expressions  using 
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Passive 

Participating 
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Remember 

Comprehension 

Understand 

AppUcation 

Apply 

Analysis 
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Synthesis 

Evaluate 

Evaluate 

Create 

Active 

and 

passive 

Active 

Karnaugh  map  (K-map). 

a.  To  develop  K-maps  with  different  size  of  input 
variables  (1  to  4). 

b.  To  map  SOP  expressions  on  K-maps. 

c.  To  develop  minimized  expressions  from  K- 
maps. 

d.  To  construct  K-map  from  a non-structured  SOP 
expression. 

16.  To  examine  the  use  of  Flip-Flops  in 
practical  applications. 

a.  To  design  parallel  data  storage  using  Flip-Flops. 

b.  To  implement  frequency  division  using  Flip- 
Flops. 

c.  To  design  binary  counter  using  Flip-Flops. 

C.  Conduct,  analyzes,  and 
interprets  experiments;  apply 
experimental  results  to  improve 
processes. 

J Factual 
Knowledge 

J Conceptual 
Knowledge 

Active 

13.  To  analyze  digital  systems  using 
combinational  logic. 

a.  To  design  a combinational  logic  system  for  a 
given  problem. 

b.  To  design  a logic  circuit  using  standard  logic 

Gen  Ed  Goals  - Students: 
b.  develop  an  ability  to  use 
modes  of  inquiry  across  a 
variety  of  disciplines  in  the 
humanities  and  the  arts,  the 
physical  sciences  and 
mathematics,  and  social 
sciences. 

b.iv.  demonstrate  an  ability 
to  use  scientific  methods  and 
theories  to  understand  the 
phenomena  studied  in  the 
natural  and  social  sciences. 

J Procedural 
Knowledge 

J Meta-Cognitive 
Knowledge 

Active 

gates  from  a given  Boolean  expression. 

c.  To  design  a logic  circuit  from  a truth  table. 

d.  To  design  logic  circuit  only  with  NAND  or 
NOR  gates. 

e.  To  analyze  the  operation  of  a combinational 
logic  circuit  with  pulse  inputs. 

f.  To  develop  Boolean  expression  from  a given 
logic  circuit 

14.  To  evaluate  combinational  logic  circuits  for 
commonly  used  digital  functionalities:  Half- 
adders and  full-adders,  parallel  binary  adders, 
comparators,  BCD  to  decimal  decodes,  BCD  to 
7-segment  decoders,  encoders,  multiplexers, 
and  demultiplexers, 
a.  To  examine  their  design  principles 

1 
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Application 

Apply 

Analysis 

Analyze 

Synthesis 

Evaluate 

Evaluate 

Create 

Passive 

b.  To  develop  combinational  logic  circuits  using 
commercially  available  ICs  to  implement  these 
common  digital  functionalities. 

c.  To  analyze  the  design  of  a magnitude 
comparator. 

d.  To  analyze  the  function  of  a decoder. 

e.  To  design  and  develop  higher  size  decoder  using 
smaller  size  decoder  ICs. 

f.  To  explain  the  use  of  BCD-7-Segment  decoder 
for  a real-life  application. 

g.  To  explain  the  design  of  encoders  using 
commercial  ICs. 

h.  To  analyze  the  function  of  an  encoder. 

i.  To  evaluate  the  operation  of  multiplexers  and 
their  implementation  using  commercially  available 
ICs. 

j.  To  evaluate  the  operation  of  demultiplexers  and 
their  implementation  using  commercially  available 
ICs. 

17.  To  design  applications  using  the  555  Timer 

a.  To  use  555  timer  for  monostable  operation. 

b.  To  use  555  timer  for  bistable  operation. 

c.  To  use  555  timer  for  astable  operation. 

Active 

18.  To  design  and  study  of  counter  applications 
using  Flip-Flops. 

a.  To  design  and  analyze  asynchronous  binary 
counters. 

b.  To  design  and  analyze  asynchronous  decade 
counter. 

c.  To  design  and  analyze  synchronous  binary 
counter. 

d.  To  design  and  analyze  synchronous  BCD 
decade  counter 

e.  To  design  and  analyze  up/down  synchronous 
counter 
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Passive 

Participating 

Active 

Knowledge 

Remember 

Comprehension 

Understand 

Application 

Apply 

Analysis 

Analyze 

Synthesis 

Evaluate 

Evaluate 

Create 

Active 

19.  To  design  and  study  of  various  shift  register 
applications. 

a.  To  demonstrate  the  use  of  D Flip-Flop  as  a shift 
register. 

b.  To  design  and  develop  serial  In/  serial  Out  shift 
register. 

c.  To  design  and  develop  serial  In/  parallel  Out 
shift  register. 

d.  To  design  and  develop  parallel  In/  serial  Out 
shift  register. 

e.  To  design  and  develop  parallel  In/  parallel  Out 
shift  register. 

f.  To  design  and  develop  bi-directional  shift 
register. 

D.  Ability  to  apply  creativity  in 
the  design  of  systems, 
components,  or  processes 
appropriate  to  program 
objectives. 

NIU  Gen  Ed  Goals  - 
Students: 

c.  develop  an  understanding 
of  the  relatedness  of  various 
disciplines  by  integrating 
knowledge  from  several 
disciplines  and  applying  that 
knowledge  to  an 
understanding  of  important 
problems  and  issues. 

J Factual 
Knowledge 

J Conceptual 
Knowledge 

J Procedural 
Knowledge 

J Meta-Cognitive 
Knowledge 

Active 

14.  To  evaluate  combinational  logic  circuits  for 
commonly  used  digital  functionalities:  Half- 
adders and  full-adders,  parallel  binary  adders, 
comparators,  BCD  to  decimal  decodes,  BCD  to 
7-segment  decoders,  encoders,  multiplexers, 
and  demultiplexers. 

a.  To  examine  their  design  principles 

b.  To  develop  combinational  logic  circuits  using 
commercially  available  ICs  to  implement  these 
common  digital  functionalities. 

c.  To  analyze  the  design  of  a magnitude 
comparator. 

d.  To  analyze  the  function  of  a decoder. 

e.  To  design  and  develop  higher  size  decoder  using 
smaller  size  decoder  ICs. 

f.  To  explain  the  use  of  BCD-7-Segment  decoder 
for  a real-life  application. 

g.  To  explain  the  design  of  encoders  using 
commercial  ICs. 

h.  To  analyze  the  function  of  an  encoder. 

i.  To  evaluate  the  operation  of  multiplexers  and 
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Create 

their  implementation  using  commercially  available 
ICs. 

j.  To  evaluate  the  operation  of  demultiplexers  and 
their  implementation  using  commercially  available 
ICs. 

Active 

18.  To  design  and  study  counter  applications 
using  Flip-Flops. 

a.  To  design  and  analyze  asynchronous  binary 
counters. 

b.  To  design  and  analyze  asynchronous  decade 
counter. 

c.  To  design  and  analyze  synchronous  binary 
counter. 

d.  To  design  and  analyze  synchronous  BCD 
decade  counter 

e.  To  design  and  analyze  up/down  synchronous 
counter 

Active 

19.  To  design  and  study  various  shift  register 
applications. 

a.  To  demonstrate  the  use  of  D Flip-Flop  as  a shift 
register. 

b.  To  design  and  develop  serial  In/  serial  Out  shift 
register. 

c.  To  design  and  develop  serial  In/  parallel  Out 
shift  register. 

d.  To  design  and  develop  parallel  In/  serial  Out 
shift  register. 

e.  To  design  and  develop  parallel  In/  parallel  Out 
shift  register. 

f.  To  design  and  develop  bi-directional  shift 
register. 
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Create 

E. 

te 

Function  effectively  on 
ams. 

NIU  Gen  Ed  Goals- 
Students: 
b.iii.  demonstrate  a 
knowledge  of  cultural 
traditions  and  philosophical 
ideas  that  have  shaped 
societies,  civilizations,  and 
human  self-conceptions, 
d.  develop  social 
responsibility  and 
preparation  for  citizenship 
through  global  awareness, 
environmental  sensitivity, 
and  an  appreciation  of 
cultural  diversity. 

Factual  Knowledge 

Conceptual 

Knowledge 

Procedural 

Knowledge 

Meta-Cognitive 

Knowledge 

F.  Identify,  analyze,  and  solve 
technical  problems. 

Gen  Ed  Goals  - Students: 
b.  develop  an  ability  to  use 
modes  of  inquiry  across  a 
variety  of  disciplines  in  the 
humanities  and  the  arts,  the 
physical  sciences  and 
mathematics,  and  social 
sciences. 

b.iv.  demonstrate  an  ability 
to  use  scientific  methods  and 
theories  to  understand  the 
phenomena  studied  in  the 
natural  and  social  sciences. 

J Factual 
Knowledge 

J Conceptual 
Knowledge 

J Procedural 
Knowledge 

J Meta-Cognitive 
Knowledge 

Passive 

and 

active 

Active 

and 

passive 

7.  To  analyze  the  performance  characteristics 
and  parameters  for  logic  gates  and  evaluate 
their  significance  in  digital  design. 

a.  To  estimate  the  propagation  delay  for  a given 
logic  gate  and  realize  its  significance  in  digital 
design. 

b.  To  estimate  the  speed-power  product  as  a 
measure  of  the  performance  of  a logic  circuit. 

c.  To  estimate  fan-out  and  loading  conditions 
while  designing  a logic  circuit. 

d.  To  interpret  data  sheets  for  different  logic  gate 
ICs. 

e.  To  evaluate  data  sheet  information  while 
making  design  decisions. 

9.  To  analyze  digital  logic  circuits  using  Boolean 
algebra. 

a.  To  develop  a Boolean  expression  for  a given 
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Evaluate 

Evaluate 

Create 

Active 

Active 

logic  circuit. 

b.  To  evaluate  a Boolean  expression  and  prepare  a 
truth-table  for  the  logic  circuit. 

c.  To  demonstrate  the  use  of  Boolean  algebra 
while  minimizing  Boolean  expressions. 

13.  To  analyze  digital  systems  using 
combinational  logic. 

a.  To  design  a combinational  logic  system  for  a 
given  problem. 

b.  To  design  a logic  circuit  using  standard  logic 
gates  from  a given  Boolean  expression. 

c.  To  design  a logic  circuit  from  a truth  table. 

d.  To  design  logic  circuit  only  with  NAND  or 
NOR  gates. 

e.  To  analyze  the  operation  of  a combinational 
logic  circuit  with  pulse  inputs. 

f.  To  develop  Boolean  expression  from  a given 
logic  circuit 

14.  To  evaluate  combinational  logic  circuits  for 
commonly  used  digital  functionalities:  Half- 
adders and  full-adders,  parallel  binary  adders, 
comparators,  BCD  to  decimal  decodes,  BCD  to 
7-segment  decoders,  encoders,  multiplexers, 
and  demultiplexers. 

a.  To  examine  their  design  principles 

b.  To  develop  combinational  logic  circuits  using 
commercially  available  ICs  to  implement  these 
common  digital  functionalities. 

c.  To  analyze  the  design  of  a magnitude 
comparator. 

d.  To  analyze  the  function  of  a decoder. 

e.  To  design  and  develop  higher  size  decoder  using 
smaller  size  decoder  ICs. 

f.  To  explain  the  use  of  BCD-7-Segment  decoder 
for  a real-life  application. 

1 
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g.  To  explain  the  design  of  encoders  using 
commercial  ICs. 

h.  To  analyze  the  function  of  an  encoder. 

i.  To  evaluate  the  operation  of  multiplexers  and 
their  implementation  using  commercially  available 
ICs. 

j.  To  evaluate  the  operation  of  demultiplexers  and 
their  implementation  using  commercially  available 
ICs. 

1 

1 

im 

Active 

18.  To  design  and  study  counter  applications 
using  Flip-Flops. 

a.  To  design  and  analyze  asynchronous  binary 
counters. 

b.  To  design  and  analyze  asynchronous  decade 
counter. 

c.  To  design  and  analyze  synchronous  binary 
counter. 

d.  To  design  and  analyze  synchronous  BCD 
decade  counter 

e.  To  design  and  analyze  up/down  synchronous 
counter 

1 

Active 

19.  To  design  and  study  various  shift  register 
applications. 

a.  To  demonstrate  the  use  of  D Flip-Flop  as  a shift 
register. 

b.  To  design  and  develop  serial  In/  serial  Out  shift 
register. 

c.  To  design  and  develop  serial  In/  parallel  Out 
shift  register. 

d.  To  design  and  develop  parallel  In/  serial  Out 
shift  register. 

e.  To  design  and  develop  parallel  In/  parallel  Out 
shift  register. 

f.  To  design  and  develop  bi-directional  shift 
register. 
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Understand 

Application 

Apply 
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Evaluate 

Create 

G.  Communicate  effectively  in 
writing. 

NIU  Gen  Ed  Goals-  Students: 
a.  develop  habits  of  writing, 
speaking,  and  reasoning  necessary 
for  continued  learning, 
a.i.  communicate  clearly  in  written 
English,  demonstrating  ability  to 
comprehend,  analyze,  and 
interrogate  critically. 

Factual  Knowledge 

-iConceptual 

Knowledge 

Procedural 

Knowledge 

Meta-Cognitive 

Knowledge 

H.  Communicate  effectively 
orally. 

NIU  Gen  Ed  Goals  - Students: 
a.  develop  habits  of  writing, 
speaking,  and  reasoning 
necessary  for  continued 
learning. 

a.i.  communicate  clearly  in 
written  English,  demonstrating 
ability  to  comprehend,  analyze, 
and  interrogate  critically, 
aii.  communicate  in  a manner 
that  unites  theory,  criticism,  and 
practice  in  speaking  & writing. 

Factual  Knowledge 

Conceptual 

Knowledge 

Procedural 

Knowledge 

Meta-Cognitive 

Knowledge 
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Bloom’s  Cognitive  Process  Dimension 

Passive 

Participating 

Active 

Knowledge 

Remember 

Comprehension 

Understand 

AppUcation 

Apply 

Analysis 

Analyze 

Synthesis 

Evaluate 

Evaluate 

Create 

I.  Recognize  the  need  for,  and  an 
ability  to  engage  in  life  long 
learning. 

J Factual 
Knowledge 

J Conceptual 
Knowledge 

J Procedural 

Active 

13.  To  analyze  digital  systems  using 
combinational  logic. 

a.  To  design  a combinational  logic  system  for  a 
given  problem. 

NIU  Gen  Ed  Goals  - Students: 
a.iv.  are  able  to  access  and  use 
various  information  sources. 

b.  To  design  a logic  circuit  using  standard  logic 
gates  from  a given  Boolean  expression. 

c.  To  design  a logic  circuit  from  a truth  table. 

Internet,  text,  and  field  case. 

Knowledge 

d.  To  design  logic  circuit  only  with  NAND  or 
NOR  gates. 

e.  To  analyze  the  operation  of  a combinational 
logic  circuit  with  pulse  inputs. 

f.  To  develop  Boolean  expression  from  a given 
logic  circuit 

14.  To  evaluate  combinational  logic  circuits  for 
commonly  used  digital  functionalities:  Half- 
adders and  full-adders,  parallel  binary  adders, 
comparators,  BCD  to  decimal  decodes,  BCD  to 
7-segment  decoders,  encoders,  multiplexers, 
and  demultiplexers. 

a.  To  examine  their  design  principles 

b.  To  develop  combinational  logic  circuits  using 
commercially  available  ICs  to  implement  these 
common  digital  functionalities. 

c.  To  analyze  the  design  of  a magnitude 
comparator. 

d.  To  analyze  the  function  of  a decoder. 

e.  To  design  and  develop  higher  size  decoder  using 
smaller  size  decoder  ICs. 

f.  To  explain  the  use  of  BCD-7-Segment  decoder 
for  a real-life  application. 

g.  To  explain  the  design  of  encoders  using 
commercial  ICs. 

h.  To  analyze  the  function  of  an  encoder. 

i.  To  evaluate  the  operation  of  multiplexers  and 
their  implementation  using  commercially  available 
ICs. 

J Meta-Cognitive 
Knowledge 

Active 
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Table  B.7.b.2:  AbulAzad  May  15‘''  2006 


ABET/TAC/NAIT 
Engineering  & Technology 
Outcomes 

Bloom's 

Knowledge 

Dimension 

Dale’s  Cone 

Student  Learning  Objectives 

Bloom’s  Cognitive  Process  Dimension 

Passive 

Participating 

Active 

Knowledge 

Remember 

Comprehension 

Understand 

AppUcation 

Apply 

Analysis 

Analyze 

Synthesis 

Evaluate 

Evaluate 

Create 

j.  To  evaluate  the  operation  of  demultiplexers  and 
their  implementation  using  commercially  available 
ICs 

Active 

18.  To  design  and  study  counter  applications 
using  Flip-Flops. 

a.  To  design  and  analyze  asynchronous  binary 
counters. 

b.  To  design  and  analyze  asynchronous  decade 
counter. 

c.  To  design  and  analyze  synchronous  binary 
counter. 

d.  To  design  and  analyze  synchronous  BCD 
decade  counter 

e.  To  design  and  analyze  up/down  synchronous 
counter 

Active 

19.  To  design  and  study  various  shift  register 
applications. 

a.  To  demonstrate  the  use  of  D Flip-Flop  as  a shift 
register. 

b.  To  design  and  develop  serial  In/  serial  Out  shift 
register. 

c.  To  design  and  develop  serial  In/  parallel  Out 
shift  register. 

d.  To  design  and  develop  parallel  In/  serial  Out 
shift  register. 

e.  To  design  and  develop  parallel  In/  parallel  Out 
shift  register. 

f.  To  design  and  develop  bi-directional  shift 
register. 
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Table  B.7.b.2:  AbulAzad  May  15‘''  2006 


ABET/TAC/NAIT 
Engineering  & Technology 
Outcomes 

Bloom's 

Knowledge 

Dimension 

Dale’s  Cone 

Student  Learning  Objectives 

Bloom’s  Cognitive  Process  Dimension 

Passive 

Participating 

Active 

Knowledge 

Remember 

Comprehension 

Understand 

AppUcation 

Apply 

Analysis 

Analyze 

Synthesis 

Evaluate 

Evaluate 

Create 

J.  Understand  professional, 
ethical,  and  social  responsibilities. 

Factual  Knowledge 

NIU  Gen  Ed  Goals  - Students: 
d.  develop  social  responsibility  and 
preparation  for  citizenship  through 
global  awareness,  environmental 
sensitivity,  and  an  appreciation  of 
cultural  diversity. 

Conceptual 

knowledge 

'rocedural 

knowledge 

leta-Cognitive 

Knowledge 

K.  Respect  for  diversity  and  a 
knowledge  of  contemporary 
professional,  societal,  and  global 
issues. 

Factual  Knowledge 

Conceptual 

Knowledge 

NIU  Gen  Ed  Goals  - Students: 
d.  develop  social  responsibility  and 
preparation  for  citizenship  through 
global  awareness,  environmental 
sensitivity,  and  an  appreciation  of 
cultural  diversity. 

'rocedural 

knowledge 

leta-Cognitive 

knowledge 

U.  Commitment  to  quality, 
timeliness,  and  continuous 

Factual  Knowledge 
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Table  B.7.b.2:  AbulAzad  May  15‘''  2006 


ABET/TAC/NAIT 
Engineering  & Technology 
Outcomes 

Bloom's 

Knowledge 

Dimension 

Dale’s  Cone 

Student  Learning  Objectives 

Bloom’s  Cognitive  Process  Dimension 

Passive 

Participating 

Active 

Knowledge 

Remember 

Comprehension 

Understand 

Application 

Apply 

Analysis 

Analyze 

Synthesis 

Evaluate 

Evaluate 

Create 

improvement. 

Conceptual 

Knowledge 

Procedural 

Knowledge 

Meta-Cognitive 

Knowledge 

M.  Ability  to  program  computers 
and/or  utilize  computer 
applications  effectively. 

Factual  Knowledge 

Conceptual 

Knowledge 

Procedural 

Knowledge 

Meta-Cognitive 

Knowledge 

N.  Ability  to  use  modern 
laboratory  techniques,  skills, 
and/or  equipment  effectively. 

Factual  Knowledge 

Conceptual 

Knowledge 

Procedural 

Knowledge 

Meta-Cognitive 

Knowledge 
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Table  B.7.b.2:  AbulAzad 


May  15'’^  2006 


ABET/TAC/NAIT 
Engineering  & Technology 
Outcomes 

Bloom's 

Knowledge 

Dimension 

Dale’s  Cone 

Student  Learning  Objectives 

Bloom’s  Cognitive  Process  Dimension 

Passive 

Participating 

Active 

Knowledge 

Remember 

Comprehension 

Understand 

Application 

Apply 

Analysis 

Analyze 

Synthesis 

Evaluate 

Evaluate 

Create 

O.  Ability  to  manage  projects 
effectively. 

NIU  Gen  Ed  Goals  - Students: 
c.  develop  an  understanding  of 
the  interrelatedness  of  various 
disciplines  by  integrating 
knowledge  from  several 
disciplines  and  applying  that 
knowledge  to  an  understanding 
of  important  problems  and 

Factual  Knowledge 

Conceptual 

Knowledge 

Procedural 

Knowledge 

Meta-Cognitive 

Knowledge 

P.  Ability  to  design,  manipulate, 
and  manage  industrial  systems. 

NIU  Gen  Ed  Goals  - Students: 
c.  develop  an  understanding  of 
the  interrelatedness  of  various 
disciplines  by  integrating 
knowledge  from  several 
disciplines  and  applying  that 
knowledge  to  an  understanding 
of  important  problems  and 

Factual  Knowledge 

Conceptual 

Knowledge 

Procedural 

Knowledge 

Meta-Cognitive 

Knowledge 
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Table  B.7.b.2:  AbulAzad 


Q.  Ability  to  manage  or  lead  Factual  Knowledge 

personnel  effectively. 

Conceptual 

Knowledge 

Procedural 

Knowledge 

Meta-Cognitive 

Knowledge 

Note:  Additional  educational  outcomes  -NIU  General  Education  Goals  - articulated  by  the  overall  program 
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May  15'’^  2006 


ABET  Engineering 

Bloom's 

Dale’s 

Student  Learning  Objectives 

Bloom’s  Cognitive  Process  Dimension 

Outcomes 

Knowledge 

Cone  of 

Dimension 

Learning 

Abhijit  Gupta 

Passive 

Knowledge 

Comprehension 

Application 

Analysis 

Synthesis 

Evaluate 

MEE  321 

Active 

Remember 

Understand 

Apply 

Analyze 

Evaluate 

Create 

A.  Apply  knowledge  of 

J Factual 

Active 

1 B.  a Define  degrees  of  Freedom 

* 

* 

* 

math,  science, 

Knowledge 

engineering 

i/Conceptual 

Knowledge 

1 B b Solve  Particle  Kinematics  problems  for  i)  Velocity 

* 

* 

* 

* 

analysis,  and  ii)  Acceleration  analysis 

J Procedural 

1 B c Solve  Rigid  Body  Kinematics  problems  for 

* 

* 

* 

* 

Knowledge 

i)  Relative  Velocity  analysis  ii)  Relative  Acceleration 

NIU  Gen  Ed 

J Meta- 

analysis 

Goals  - Students: 
a.iii.  perform 

Cognitive 

Knowledge 

1 B d Decide  how  to  choose  Particle  and/or  Rigid  Body 

* 

* 

* 

* 

basic 

formulation 

computations, 
display  facility 
with  use  of 

1 B e Solve  the  kinetics  problem 

* 

* 

formal  and 

i)  Identify  method  of  solution  by  identifying  list  of 

quantitative 

variables 

* 

* 

reasoning 

ii)  Draw  FBD  and  MAD  to  solve  for  instantaneous 

analysis  and 
problem  solving, 

forces/accelerations 

and  interpret 
mathematical 

5 a i)  Identify  the  appropriate  2DOF 

* 

* 

models  and 
statistical 

ii)  Construct  FBD  and  MAD  in  terms  of  the  chosen  2DOF 

information. 

5 b Use  the  FBD  and  MAD  to  derive  equations  of  motion 

* 

* 

7 a Derive  equations  for  MDOF  systems  & solve  for 
natural  frequencies  and  mode  shapes  in  closed  form. 

* 

* 

7 b i)  Check  orthogonalilty  of  mode  shapes  with  respect 
to  mass  and  stiffness  matrices 

ii)  Use  the  orthogonality  to  decouple  equations  of  motion. 

* 

* 

iii)  Compute  mass  normalized  mode  shapes. 

* 
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ABET  Engineering 
Outcomes 

Bloom's 

Knowledge 

Dimension 

Dale’s 
Cone  of 
Learning 

Student  Learning  Objectives 

Bloom’s  Cognitive  Process  Dimension 

Abhijit  Gupta 
MEE  321 

Passive 

Active 

Knowledge 

Remember 

Comprehension 

Understand 

Application 

Apply 

Analysis 

Analyze 

Synthesis 

Evaluate 

Evaluate 

Create 

7 e i)  Decouple  equations  of  motion  using  separation  of 
variables 

* 

9 c Solve  for  eigenvalues  (square  of  natural  frequencies) 
and  eigenvectors  (mode  shapes)  of  undamped  2DOF  and 
MDOF  systems  using  eig  command 

9 d Solve  equation  of  motion  for  a 2DOF  system  such  as 
automobile  using  state  space  method. 

* 

8 a Use  function  generator  to  generate  a periodic  wave  and 
use  the  analyzer  to  measure  the  Fourier  components 

* 

* 

* 

8 b i)  Measure  natural  frequency  of  a SDOF  system  using 
impact  hammer,  accelerometer,  and  FFT  analyzer 

ii)  Measure  damping  ratio  using  half  power  points 

iii)  Understand  various  sources  of  error  including  digital 
signal  processing  issues  and  effect  of  sensors 

* 

* 

* 

* 
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ABET  Engineering 
Outcomes 

Abhijit  Gupta 
MEE  321 


Bloom's 

Knowledge 

Dimension 


Dale’s 
Cone  of 
Learning 
Passive 
Active 


Student  Learning  Objectives 


Bloom’s  Cognitive  Process  Dimension 


Knowledge 

Remember 


Comprehension 

Understand 


Application  Analysis 
Apply  Analyze 


Synthesis 

Evaluate 


Evaluate 

Create 


B.  Design  and  conduct  J Factual 

experiments;  analyze  Knowledge 

and  interpret  data 


Active 


J Conceptual 
Knowledge 


NIU  Gen  Ed 
Goals  - 
Students: 
a.  Hi.  perform 
basic 

computations 
, display 
facility  with 
use  of  formal 
and 

quantitative 

reasoning 

analysis  and 

problem 

solving,  and 

interpret 

mathematical 

models  and 

statistical 

information. 


J Procedural 
Knowledge 

J Meta- 

Cognitive 

Knowledge 


8 c i)  Measure  natural  frequencies  and  mode  shapes  of  a 
2DOF  system  using  impact  hammer,  accelerometer,  and 
FFT  analyzer. 

ii)  Use  imaginary  part  of  transfer  functions  to  obtain 
modal  parameters. 

iii)  Obtain  damping  ratios 


* 


* 


* 
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* 

* 


* 


* * * 


NIU  Gen  Ed 
Goals  - Students: 
c.  develop  an 
understanding  of 
the  relatedness  of 
various 
disciplines  by 
integrating 
knowledge  from 
several  disciplines 
and  applying  that 
knowledge  to  an 
understanding  of 
important 
problems  and 
issues. 


C.  Design  a system, 
component,  process  to 
meet  desired  needs 
within  realistic 
constraints,  e.g. 
economic, 

environmental,  social, 
political,  ethical,  health, 
safety, 

manufacturability,  & 
sustainability. 


J Factual 
Knowledge 

J Conceptual 
Knowledge 

J Procedural 
Knowledge 

J Meta- 

Cognitive 

Knowledge 


Active  4 f Design  systems  for  desired  vibration  isolation 

10  a i)  Formulate  equation  of  motion  of  SDOF  system  and 
suggest  appropriate  stiffness/damping  for  desired  vibration 
isolation 

ii)  Formulate  equation  of  motion  of  2DOF  systems  and 
suggest  appropriate  stiffness/damping  for  desired  vibration 
isolation 

10  b Design  a tuned  absorber  system  for  vibration 
prevention  at  a resonant  forcing  frequency  while  satisfying 
design  constraints  such  as  maximum  displacement 


* * 

* * 

* * 
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D.  Function  on  multi- 
disciplinary teams. 


J Factual 
Knowledge 


Active 


NIU  Gen  Ed 
Goals- 
Students: 
b.iii. 

demonstrate  a 
knowledge  of 
cultural 
traditions  and 
philosophical 
ideas  that  have 
shaped 
societies, 
civilizations, 
and  human 
self- 
conceptions, 
d.  develop 
social 

responsibility 

and 

preparation  for 
citizenship 
through  global 
awareness, 
environmental 
sensitivity,  and 
an  appreciation 
of  cultural 
diversity. 


J Conceptual 
Knowledge 

J Procedural 
Knowledge 

J Meta- 

Cognitive 

Knowledge 


8 b i)  Measure  natural  frequency  of  a SDOF  system  using 
impact  hammer,  accelerometer,  and  FFT  analyzer 

ii)  Measure  damping  ratio  using  half  power  points 

iii)  Understand  various  sources  of  error  including  digital 
signal  processing  issues  and  effect  of  sensors 

8 c i)  Measure  natural  frequencies  and  mode  shapes  of  a 
2DOF  system  using  impact  hammer,  accelerometer,  and 
FFT  analyzer. 

ii)  Use  imaginary  part  of  transfer  functions  to  obtain 
modal  parameters. 

iii)  Obtain  damping  ratios 


* 


* 


* 


* 


* 


* 


* 
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E.  Identify,  formulate, 
and  solve  engineering, 
problems 


Gen  Ed  Goals  - 
Students: 

b.  develop  an  ability 
to  use  modes  of 
inquiry  across  a 
variety  of 
disciplines  in  the 
humanities  and  the 
arts,  the  physical 
sciences  and 
mathematics,  and 
social  sciences, 
b.iv.  demonstrate  an 
ability  to  use 
scientific  methods 
and  theories  to 
understand  the 
phenomena  studied 
in  the  natural  and 
social  sciences. 


J Factual 
Knowledge 

Active 

lA.  a Discuss  common  vibration  phenomenon 

* 

* 

* 

J Conceptual 
Knowledge 

i)  Identify  if  vibration  is  Deterministic  and  classify  as 
a)Sinusoidal,  b)  Periodic,  and  c)  Transient 

* 

* 

* 

J Procedural 

ii)  Identify  if  vibration  is  Random  and  classify  as  a) 

* 

* 

* 

Knowledge 

Stationary  and  b)  Non  Stationary 

J Meta- 

Cognitive 

Knowledge 

1 A b Identify  the  source  of  excitation  as  i)  Free  or 
ii)Forced  Vibration 

* 

* 

1 A c Identify  possible  source  of  energy  loss  and  classify 
as  i)  Undamped  or 
ii)  Damped 

* 

* 

* 

1 A d Classify  the  system  as  i)  linear  or  ii)  nonlinear 

* 

* 

* 

1 B a Define  degrees  of  Freedom 

* 

* 

* 

1 C i)  Convert  a complex  system  to  simple  sub-systems 

* 

* 

ii)  Draw  the  Schematic  of  the  sub-systems 

* 

1 D a Compute  equivalent  stiffness  for  i)  springs  in  series 
ii)  springs  in  parallel,  or  iii)  combined  effect  in  a SDOF 

* 

* 

system 

* 

1 D b Compute  equivalent  mass 

1 E a Define  basic  vibration  terminology  for  sinusoidal 
motion 

* 

* 

i)  Obtain  Fourier  series  expansion  for  periodic  motion 

ii)  Reconstruct  a periodic  wave  from  first  few  harmonics 
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2 A a Identify  the  degree  of  freedom 

* 

* 

2 A b i)  Derive  equation  of  motion  for  undamped 
translation  system 

* 

ii)  Solve  the  differential  equation  of  motion  and 
eompute  natural  frequency 

* 

2 A c i)  Derive  equation  of  motion  for  undamped 
rotational  system 

* 

ii)  Solve  the  differential  equation  of  motion  and  compute 
natural  frequency 

* 

2 B a i)  Use  compound  pendulum  for  solving  moment  of 

inertia 

ii)  Compute  center  of  percussion  and  use  it  for  sports 
applications 

2 C a Compute  equivalent  mass  and  system  and  check  if 
system  is  stable 

* 

3.  A a Derive  equation  of  motion  of  a viscously  damped 
SDOF  system 

* 

* 

3.  A.  b Compute  Critical  Damping  Constant  and  Damping 
ratio 

* 

* 

3.  A.  c Solve  for  the  response  of  i)  underdamped, 
ii)critically  damped  and  iii)  overdamped  system  due  to 
given  initial  conditions 

* 

* 
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3 A.  d Compare  the  undamped  and  damped  natural 
frequencies  and  understand  its  relevance  in  terms  of 
comparison  of  theory  and  experiment 

* 

* 

3 A.  e Use  log  decrement  to  measure  damping. 

* 

* 

3 B.  a Identify  other  types  of  damping. 

* 

* 

3 B.  b Derive  the  equation  for  columb  damping 

* 

* 

3 B c Compute  energy  loss  for  hysteretic  damping 

* 

* 

4 a Derive  the  equation  of  motion  from  FBD  & MAD 

* 

* 

4 b i)Use  the  FBD  and  MAD  to  solve  for  the  steady  state 
solution  due  to  harmonic  excitation 

* 

* 

ii)  Compute  the  total  response 

* 

iii)  Estimate  damping  ratio  from  half  power  bandwidth 

* 

* 

* 

4 c Define  transmissibility  and  observe  the  effect  of 
damping  and  frequency  ratios  on  transmissibility 

* 

* 

* 

* 

4 d Solve  for  the  response  of  a system  due  to  motion  of 
base 

* 

* 

* 

4 e Solve  for  the  response  of  a system  subjected  to 
rotating  unbalance 

* 

* 

* 

5 b Use  the  FBD  and  MAD  to  derive  equations  of  motion 

* 

* 

5 c Identify  mass,  stiffness,  and  damping  matrices  from 
the  equations  of  motion 

* 

* 

5 d Solve  for  undamped  natural  frequencies  and  mode 
shapes. 

* 

* 
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5 e Solve  for  modal  properties  of  a semidefinite  systems 

* 

6 a Use  the  equation  of  motion  to  solve  for  steady  state 
response  due  to  harmonie  exeitation  direetly  by  impedanee 
method 

* 

* 

7 a Derive  equations  for  MDOF  systems  & solve  for 
natural  frequeneies  and  mode  shapes  in  elosed  form. 

* 

* 

7 b i)  Cheek  orthogonalilty  of  mode  shapes  with  respeet 
to  mass  and  stiffness  matrices 

* 

* 

ii)  Use  the  orthogonality  to  decouple  equations  of  motion. 

iii)  Compute  mass  normalized  mode  shapes 

* 

* 

7 d i)  Define  Proportional  and  non  proportional  damping 
ii)Solve  for  damping  ratios  for  the  case  of  proportional 

* 

* 

* 

damping 

7 e ii)  Solve  for  the  response  using  mode  shapes  and 

* 

* 

generalized  coordinates 

7 e iii)  Solve  response  of  a large  DOF  systems  in  terms  of 
first  few  modes  and  generalized  coordinates 

* 

* 

* 

9 d Solve  equation  of  motion  for  a 2DOF  system  such  as 
automobile  using  state  space  method. 

* 

* 

11a  Identify  various  sources  of  periodic  forces  and  use 
Fourier  analysis  to  solve  for  response  of  SDOF  system 
subjected  to  periodic  force 

* 

* 

* 

* 

11b  Identify  various  sources  of  transient  forces  and  solve 
for  response  of  SDOF  system  subjected  to  the  transient 
forces 

* 

* 

* 

66 


F.  Understand 
professional  and  ethical 
responsibility 


NIU  Gen  Ed  Goals 
- Students: 
d.  develop  social 
responsibility  and 
preparation  for 
citizenship  through 
global  awareness, 
environmental 
sensitivity,  and  an 
appreciation  of 
cultural  diversity. 


G.  Communicate 
effectively 


^/Factual 

Knowledge 


Active 


8 a Use  funetion  generator  to  generate  a periodie  wave  and 
use  the  analyzer  to  measure  the  Fourier  components 


* 


* 


* 


NIU  Gen  Ed 
Goals-  Students: 
a.  develop  habits  of 
writing,  speaking, 
and  reasoning 
necessary  for 
continued 
learning, 
a.i.  communicate 
clearly  in  written 
English, 
demonstrating 
ability  to 
comprehend, 
analyze,  and 
interrogate 
critically. 


i/Conceptual 

Knowledge 

^Procedural 

Knowledge 

VMeta- 

Cognitive 

Knowledge 


8 b i)  Measure  natural  frequency  of  a SDOF  system  using 
impaet  hammer,  accelerometer,  and  FFT  analyzer 

ii)  Measure  damping  ratio  using  half  power  points 

iii)  Understand  various  sources  of  error  ineluding  digital 
signal  processing  issues  and  effeet  of  sensors 

8 c i)  Measure  natural  frequencies  and  mode  shapes  of  a 
2DOF  system  using  impact  hammer,  accelerometer,  and 
FFT  analyzer. 

ii)  Use  imaginary  part  of  transfer  functions  to  obtain 
modal  parameters. 

iii)  Obtain  damping  ratio 
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H.  understand  impact 
of  engineering  solution 
in  a global  economic, 
environmental,  societal 
context 

NIU  Gen  Ed  Goals 
- Students: 
d.  develop  social 
responsibility  and 
preparation  for 
citizenship  through 
global  awareness, 
environmental 
sensitivity,  and  an 
appreciation  of 
cultural  diversity. 

s 

^Factual 

Knowledge 

^Conceptual 

Knowledge 

^Procedural 

Knowledge 

VMeta- 

Cognitive 

Knowledge 

Active 

10.  A.  i)  Formulate  equation  of  motion  of  SDOF  system 
and  suggest  appropriate  stiffness/damping  for  desired 
vibration  isolation 

ii)  Formulate  equation  of  motion  of  2DOF  systems  and 
suggest  appropriate  stiffness/damping  for  desired  vibration 
isolation 

B.  Design  a tuned  absorber  system  for  vibration 
prevention  at  a resonant  forcing  frequency  while  satisfying 
design  constraints  such  as  maximum  displacement 

* 

* 

* 

* 

I.  Recognize  the  need 
for,  and  have  capabilitj 
to  engage  in  life  long 
learning. 

NIU  Gen  Ed  Goals 
- Students: 
a.iv.  are  able  to 
access  and  use 
various  information 
sources. 

Internet,  text,  and 
field  case. 

J 

^Factual 

Knowledge 

^Conceptual 

Knowledge 

^Procedural 

Knowledge 

VMeta- 

Cognitive 

Knowledge 

Active 

9 A.  a)  Reconstruct  a periodic  wave  from  first  few 
harmonics  and  plot  using  Matlab 

b)  To  solve  for  complete  solution  for  response  of  a SDOF 
using  Matlab 

c) Solve  for  eigenvalues  (square  of  natural  frequencies)  and 
eigenvectors  (mode  shapes)  of  undamped  2D  OF  and 
MDOF  systems 

d)  Solve  equation  of  motion  for  a 2DOF  system  such  as 
automobile  using  state  space  method 

* 

* 

* 
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J.  knowledge  of 
contemporary  issues 


J Factual 
Knowledge 


Active 


NIU  Gen  Ed  Goals 
- Students; 
d.  develop  social 
responsibility  and 
nrenaration  for 


J Conceptual 
Knowledge 

J Procedural 
Knowledge 

J Meta- 

Cognitive 

Knowledge 


10  a i)  Formulate  equation  of  motion  of  SDOF  system  and 
suggest  appropriate  stiffness/damping  for  desired  vibration 
isolation 

ii)  Formulate  equation  of  motion  of  2DOF  systems  and 
suggest  appropriate  stiffness/damping  for  desired  vibration 
isolation 

10  b Design  a tuned  absorber  system  for  vibration 
prevention  at  a resonant  forcing  frequency  while  satisfying 
design  constraints  such  as  maximum  displacement 


* 


* 


* 


* 


* 


* 


* 


* 


* 


* 


* 


* 


11c  Define  response  spectrums  and  discuss  their  use  in 
structural  design 


K.  use  techniques, 
skills,  and  modern 
engineering  tools 
necessary  for 
engineering  practice 


J Factual 
Knowledge 


Active 


J Conceptual 
Knowledge 


6 b Use  Matlab  to  solve  the  equations  of  motion  directly 
and  by  state  space  method 

7 c Solve  for  natural  frequencies  and  mode  shapes  by 
Matlab 


* 


* 


* 


* 


* 


* 


* 


* 


* 


J Procedural 
Knowledge 

J Meta- 

Cognitive 

Knowledge 


9 a Reconstruct  a periodic  wave  from  first  few 
harmonics  and  plot  using  Matlab 
9 b To  solve  for  complete  solution  for  response  of  a 
SDOF  using  Matlab 


* 


* 


* 


* 


* 


9 c Solve  for  eigenvalues  (square  of  natural  frequencies) 
and  eigenvectors  (mode  shapes)  of  undamped  2DOF  and 
MDOF  systems  using  eig  command 


Addition:  Additional  educational  outcomes  -NIU  General  Education  Goals  - articulated  by  the  overall  program 
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Table  B.7.b.3:  Technology  496  Industrial  Project  Management  - Senior  Design  Capstone  (Scarborough,  2005) 

ABET/TAC/NAIT  Bloom's  Knowledge  Dale’s  Student  Learning 

Engineering  & Technology  Dimension  Cone  Outcomes 


Bloom's  Cognitive  Process  Dimension 

Knowledge  Comprehension  Application 

Remember  Understand  Apply 


= range  of  cognitive  processing 
J = yes  on  if  Bloem's  knowledge  cetegory  is  met 


Analysis  Synthesis  Evaluate 

Analyze  Evaluate  Create 


A.  Mastery  of  knowledge,  techniques,  J Factual  Knowledge:  Active 

skills,  modern  tools  of  disciplines. 

J Conceptual  Knowledge: 

J Procedural  Knowledge: 

J Meta-cognitive  Knowledge: 

J Factual  Knowledge:  Active 

J Conceptual  Knowledge: 

J Procedural  Knowledge: 

J Meta-cognitive  Knowledge: 


B.  Apply  current  knowledge  and 
adapt  to  emerging  applications 
of  math,  science,  engineering,  and 
technology. 


NIU  Gen  Ed  Goals  - Students: 
a.iii.  perform  basic  computations, 
display  facility  with  use  of  formal 
and  quantitative  reasoning  analysis 
and  problem  solving,  and  interpret 
mathematical  models  and 
statistical  information. 


6.  To  demonstrate  effective  project: 

a.  planning 

b.  initiation 

c.  execution 

d.  termination 


8.  To  integrate  mathematics,  the  sciences, 
communication,  management,  technical, 
and  technological  knowledge  and  skills 
to  accomplish  team  and  project  objectives. 


C.  Conduct,  analyze,  and  interpret  Factual  Knowledge: 

experiments;  apply  experimental 

results  to  improve  processes.  Conceptual  Knowledge: 


Gen  Ed  Goals  - Students:  Procedural  Knowledge: 

b.  develop  an  ability  to  use  modes 

of  inquiry  across  a variety  of  Meta-cognitive  Knowledge: 

disciplines  in  the  humanities  and 

the  arts,  the  physical  sciences 

and  mathematics,  and  social 

sciences. 

b.iv.  demonstrate  an  ability  to 
use  scientific  methods  and 
theories  to  understand  the 
phenomena  studied  in  the 
natural  and  social  sciences. 


NA 
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Engineering  & Technology 


Bloom's  Knowledge  Dale’s  Student  Learning 

Dimension  Cone  Outcomes 


Bloom's  Cognitive  Process  Dimension 

Knowledge  Comprehension  Application  Analysis  Synthesis 

Remember  Understand  Apply  Analyze  Evaluate 


D.  ability  to  apply  creativity  in  the  J Factual  Knowledge:  Active 

design  of  systems,  components, 

or  processes  appropriate  to  J Conceptual  Knowledge: 

program  objectives. 

J Procedural  Knowledge: 


NIU  Gen  Ed  Goals  - Students: 
c.  develop  an  understanding  of 
the  relatedness  of  various 
disciplines  by  integrating 
knowledge  from  several 
disciplines  and  applying  that 
knowledge  to  an  understanding 
of  important  problems  and 


J Meta-cognitive  Knowledge: 


E.  Function  effectively  on  teams. 


NIU  Gen  Ed  Goals-Students: 
b.iii.  demonstrate  a knowledge  of 
cultural  traditions  and 
philosophical  ideas  that  have 
shaped  societies,  civilizations,  and 
human  self-conceptions, 
d.  develop  social  responsibility  and 
preparation  for  citizenship  through 
global  awareness,  environmental 
sensitivity,  and  an  appreciation  of 
cultural  diversity. 


J Factual  Knowledge:  Active 

J Conceptual  Knowledge: 

J Procedural  Knowledge 
J Meta-cognitive  Knowledge: 


8.  To  integrate  mathematics,  the  sciences, 
communication,  management,  technical, 
technological  knowledge  and  skills  to 
accomplish  team  and  project  objectives. 

a.  design  a vehicle  to  technical  specifications 

b.  build  the  vehicle  to  technical  specifications 

c.  solve  technical  problems  associated  with 
design,  construction,  and  evaluation 

d.  test  and  evaluate  vehicle  against  technical 
specifications 


1.  To  identify  and  describe  major  problems, 
issues,  concerns,  and  solutions  that  relate  to 
projects,  PM,  Pteams,  and  P leaders,  international 
projects,  and  multi-cultural  teams. 

2.  To  identify  and  describe  best  practices  for  managing 
projects  and  leading  teams,  including  international 
projects  and  multi-cultural  teams. 

3.  To  perform  effectively  on  a project  team 
(multi-cultural  when  possible) 

a.  To  engage  in  conflict  resolution  to  resolve 
team  issues. 


Evaluate 

Create 


71 


ABET/TAC/NAIT 
Engineering  & Technology 


Bloom's  Knowledge  Dale’s  Student  Learning  Outcomes 

Dimension  Cone 


F.  Identify,  analyze,  and  solve 
technical  problems. 


Gen  Ed  Goals  - Students: 
b.  develop  an  ability  to  use  modes  of 
inquiry  across  a variety  of  disciplines 
in  the  humanities  and  the  arts,  the 
physical  sciences  and  mathematics, 
and  social  sciences, 
b.iv.  demonstrate  an  ability  to  use 
scientific  methods  and  theories  to 
understand  the  phenomena  studied  in 
the  natural  and  social  sciences. 


J Factual  Knowledge:  Active 

J Conceptual  Knowledge: 

J Procedural  Knowledge: 

J Meta-cognitive  Knowledge: 


8.  To  integrate  mathematics,  the  sciences, 
communication,  management,  technical, 
technological  knowledge  and  skills  to 
accomplish  team  and  project  objectives. 

a.  design  a vehicle  to  technical  specifications 

b.  build  the  vehicle  to  technical  specifications 

c.  solve  technical  problems  associated  with 
design,  construction,  and  evaluation 

d.  test,  evaluate  vehicle  against  technical 
specifications. 


G.  Communicate  effectively  in  writing. 


NIU  Gen  Ed  Goals-  Students: 
a.  develop  habits  of  writing, 
speaking,  and  reasoning  necessary 
for  continued  learning, 
a.i.  communicate  clearly  in  written 
English,  demonstrating  ability  to 
comprehend,  analyze,  and 
interrogate  critically. 


J Factual  Knowledge:  Active 

J Conceptual  Knowledge: 

J Procedural  Knowledge: 


9.  To  design,  develop,  prepare,  and  deUver 

a.  executive  team  presentation 

b.  team  portfolio 

c.  team  website 


J Meta-cognitive  Knowledge: 
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Bloom's  Cognitive  Process  Dimension 

Knowledge  Comprehension  Application  Analysis  Synthesis  Evaluate 

Remember  Understand  Apply  Analyze  Evaluate  Create 


ABET/TAC/NAIT 
Engineering  & Technology 


Bloom's  Knowledge  Dale’s 
Dimension  Cone 


Student  Learning  Outcomes 


Bloom's  Cognitive  Process  Dimension 

Knowledge  Comprehension  Application  Analysis  Synthesis 

Remember  Understand  Apply  Analyze  Evaluate 


Evaluate 

Create 


H.  Communicate  effectively  orally. 

NIU  Gen  Ed  Goals  - Students: 
a.  develop  habits  of  writing, 
speaking,  and  reasoning 
necessary  for  continued 
learning. 

a.i.  communicate  clearly  in 
written  English,  demonstrating 
ability  to  comprehend,  analyze, 
and  interrogate  critically, 
aii.  communicate  in  a manner 
that  unites  theory,  criticism,  and 
practice  in  speaking  & writing. 


I.  Recognize  the  need  for,  and  an 
ability  to  engage  in  life  long  learning. 


NIU  Gen  Ed  Goals  - Students: 
a.iv.  are  able  to  access  and  use 
various  information  sources. 
Internet,  text,  and  field  case. 


J.  Understand  professional,  ethical, 
and  social  responsibilities. 


NIU  Gen  Ed  Goals  - Students: 
d.  develop  social  responsibility  and 
preparation  for  citizenship  through 
global  awareness,  environmental 
sensitivity,  and  an  appreciation  of 
cultural  diversity. 


J Factual  Knowledge:  Active 

J Conceptual  Knowledge: 

J Procedural  Knowledge: 

J Meta-cognitive  Knowledge: 


7.  To  demonstrate  effective  project 

a.  planning 

b.  initiation 

c.  execution 

d.  termination 

9.  To  design,  develop,  prepare,  and  deliver 

a.  team  presentation 

b.  team  portfolio 

c.  team  website 


J Factual  Knowledge: 

J Conceptual  Knowledge: 
Procedural  Knowledge: 


A 

Intermediate!.  To  identify  and  describe  major  problems, 

Active  issues,  concerns,  and  solutions  that  relate  to  / 
projects,  project  management,  project  teams,  / 
and  project  leaders,  including  international  ^ 
projects  and  multi-cultural  teams. 


Meta-cognitive  Knowledge:  2.  To  identify  and  describe  best  practices  for 

managing  projects  and  leading  teams,  including 
international  projects  and  multi-cultural  teams. 


J Factual  Knowledge:  Active 

J Conceptual  Knowledge: 


5.  To  exhibit  leadership  by  engaging  in 
a team  community  service  project. 


J Procedural  Knowledge: 


J Meta-cognitive  Knowledge: 
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ABET/TAC/NAIT  Bloom's  Knowledge  Dale’s 

Engineering  & Technology  Dimension  Cone 

K.  Respect  for  diversity  and  a knowledge  VF actual  Knowledge:  Active 

of  contemporary  professional,  societal, 

and  global  issues.  ^Conceptual  Knowledge: 

^Procedural  Knowledge: 
^Meta-cognitive  Knowledge: 


NIU  Gen  Ed  Goals  - Students: 
d.  develop  social  responsibility  and 
preparation  for  citizenship  through 
global  awareness,  environmental 
sensitivity,  and  an  appreciation  of 
cultural  diversity. 


L.  Commitment  to  quality,  timeliness,  ^Factual  Knowledge:  Active 

and  continuous  improvement. 

^Conceptual  Knowledge: 
^Procedural  Knowledge: 
^Meta-cognitive  Knowledge: 


Student  Learning  Outcomes  Bloom's  Cognitive  Process  Dimension 

Knowledge  Comprehension  Application 

Remember  Understand  Apply 


3.  To  perform  effectively  on  a project  team 
(multicultural  when  possible), 
a.  To  engage  in  conflict  resolution  to  resolve 
team  issues. 


Analysis 

Analyze 


Synthesis 

Evaluate 


Evaluate 

Create 


6.  To  demonstrate  effective  project 

a.  planning 

b.  initiation 

c.  execution 

d.  termination 

3.  To  perform  effectively  on  a project  team 
(multicultural  when  possible) 

a.  To  engage  in  conflict  resolution  to  resolve  team  issues. 


8.  To  integrate  mathematics,  the  sciences 
communication,  management,  technical 
team  and  project  objectives. 

a.  design  a vehicle  to  technical  specifications 

b.  build  the  vehicle  to  technical  specifications 

c.  solve  technical  problems  associated  with 
design,  construction,  and  evaluation 

d.  test  and  evaluate  vehicle  against  technical  specifications 


M.  Ability  to  program  computers 
and/or  utilize  computer  applications 
effectively. 


VF actual  Knowledge:  Active 

^Conceptual  Knowledge: 
^Procedural  Knowledge: 
^Meta-Cognitive  Knowledge: 


6.  To  demonstrate  effective  project 

a.  planning 

b.  initiation 

c.  execution 

d.  termination 
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ABET/TAC/NAIT 
Engineering  & Technology 


N.  Ability  to  use  modern  laboratory 
techniques,  skills,  and/or  equipment 
effectively. 


Bloom's  Knowledge  Dale’s 
Dimension  Cone 


Student  Learning  Outcomes  Bloom's  Cognitive  Process  Dimension 

Knowledge  Comprehension  Application  Analysis  Synthesis  Evaluate 

Remember  Understand  Apply  Analyze  Evaluate  Create 


/Factual  Knowledge:  Active 

/Conceptual  Knowledge: 
/Procedural  Knowledge: 
/Meta-Cognitive  Knowledge: 


8.  To  integrate  mathematics,  the  sciences 
communication,  management,  technical  and  — 
technological  knowledge  and  skills  to  accomplisnN 
team  and  project  objectives. 

a.  design  a vehicle  to  technical  specifications  , 

b.  build  the  vehicle  to  technical  specifications 

c.  solve  technical  problems  associated  with  project 
design,  construction,  and  evaluation 

d.  test  and  evaluate  vehicle  against  technical  specifications 
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ABET/TAC/NAIT 
Engineering  & Technology 


Bloom's  Knowledge  Dale’s 
Dimension  Cone 


Student  Learning  Outcomes 


Bloom's  Cognitive  Process  Dimension 

Knowledge  Comprehension  Application  Analysis  Synthesis 

Remember  Understand  Apply  Analyze  Evaluate 


Evaluate 

Create 


O.  Ability  to  manage  projects 
effectively. 


NIU  Gen  Ed  Goals  - Students: 
c.  develop  an  understanding  of 
the  interrelatedness  of  various 
disciplines  by  integrating 
knowledge  from  several 
disciplines  and  applying  that 
knowledge  to  an  understanding 
of  important  problems  and 


yp actual  Knowledge:  Active 

^Conceptual  Knowledge: 
^Procedural  Knowledge: 
VMeta-cognitive  Knowledge: 


1.  To  identify  an  describe  major  problems,  issues,  concerns 
and  solutions  that  relate  to  projects,  Pmanagement,  P teami 
and  P leaders,  including  international  projects  and  leading 
multicultural  teams. 


2.  To  identify  and  describe  best  practices  for  managing  projec 
and  leading  teams,  including  international  projects  and  multi 
cultural  teams. 


3.  To  perform  effectively  on  a project  team  (multicultural  team 
when  possible). 


Active  4.  To  prepare  the  team  for  project  work  by 

a.  developing  a team  operations  manual 

b.  developing  a peer  and  team  assessment  system 

c.  creating  the  team  organization  and  process 

d.  developing  a team  project  plan 

6.  To  demonstrate  project 

a.  planning 

b.  initiation 

c.  execution 

d.  termination 

e.  problem  solving 


7.  To  demonstrate  effective  use  of  project  management  techniques 
and  tools  in  the  management  of  a technical  project. 

a.  the  development  of  a project  plan 

b.  use  of  MS  Project 

c.  use  of  appropriate  financial  planning  and  operations  procedures 

d.  use  of  appropriate  procurement  procedures 

e.  scheduling  techniques 

f.  use  of  the  MACE  process 


9.  To  design,  develop,  prepare,  and  deliver 

a.  executive  team  presentation 

b.  team  portfolio 

c.  team  website 
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ABET/TAC/NAIT 
Engineering  & Technology 


Bloom's  Knowledge  Dale’s  Student  Learning  Outcomes 
Dimension  Cone 


P.  Ability  to  design,  manipulate,  Factual  Knowledge: 

and  manage  industrial  systems. 

Conceptual  Knowledge: 
Procedural  Knowledge: 
Meta-cognitive  Knowledge: 


NIU  Gen  Ed  Goals  - Students: 
c.  develop  an  understanding  of 
the  interrelatedness  of  various 
disciplines  by  integrating 
knowledge  from  several 
disciplines  and  applying  that 
knowledge  to  an  understanding 
of  important  problems  and 


NA 


Bloom's  Cognitive  Process  Dimension 

Knowledge  Comprehension  Application 

Remember  Understand  Apply 


Analysis  Synthesis 
Analyze  Evaluate 


Evaluate 

Create 


Q.  Ability  to  manage  or  lead  personnel 
effectively. 


yp actual  Knowledge:  Active 

^Conceptual  Knowledge: 
^Procedural  Knowledge: 
VMeta-cognitive  Knowledge: 


3.  To  perform  effectively  on  a project  team 
(multicultural  when  possible). 

a.  To  engage  in  conflict  resolution  to  resolve 
team  issues. 

b.  To  engage  in  the  leadership  of  the  team, 
team  members,  or  work  package  sub-team 
members. 


4.  The  team  will  prepare  project  work  by 

a.  developing  a team  operations  manual 

b.  developing  a peer  and  team  assessment  system 

c.  creating  the  team  organization  and  process 

d.  developing  a team  project  plan 


5.  To  exhibit  leadership  while  engaged  in  a 
team  community  service  project 

a.  plan 

b.  initiate 

c.  execute 

d.  terminate 

e.  report 


Note:  additional  educational  outcomes  articulated  by  the  overall  program  **  See  in  text  boxes  above  - NIU  General  Education  Goals 
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COURSE  CONTENT  SCHEDULE  TEACHING  AND  LEARNING  ANALYSIS 
JULE  DEE  SCARBOROUGH,  PH.D. 

(See  Program  in  A. 5 and  Worksheets  in  C.l) 


Once  the  professors  had  analyzed  their  2005  courses  and  tests  and  redeveloped  the  student 
learning  outcomes  for  the  2006  courses,  they  created  a course  content  calendar  for  the  2006 
courses.  These  professors  had  never  included  a course  calendar,  noting  due  dates,  the  topics 
and  activities  for  each  course  day,  and  more,  in  their  syllabus.  That  was  a syllabus 
requirement,  so  we  created  the  new  2006  course  calendar.  However,  we  also  used  the  course 
calendar  as  a quality  check  tool.  In  addition  to  its  intended  purpose,  to  communicate  what 
was  going  on  or  expected  in  the  course  from  the  students,  each  professor  used  it  as  an 
analysis  tool  and  graphic  organizer  to  determine  if  the  newly  developed  student  learning 
outcomes  for  the  2006  courses,  designed  to  more  actively  engage  students  in  learning  and  to 
present  the  opportunity  to  use  a variety  of  teaching  models  and  styles,  did  so.  Each  professor 
took  the  legend  below  and  identified  by  course  day  which  teaching  models  and  styles  could 
be  used.  They  also  considered  and  noted  what  student  learning  styles  were  possible  for  each 
day  and,  finally,  if  they  could  expect  to  achieve  the  upper  levels  of  Bloom’s  Cognitive 
Dimension  and  if  the  learning  was  active  according  to  Dale’s  Cone  of  Learning.  This  gave 
the  professors  a visual  or  graphic  organizer  of  the  teaching  and  learning  context  for  the 
course,  using  the  actual  course  calendar  to  be  included  on  the  syllabus  later  when  developing 
the  whole  syllabus  during  that  program  component.  The  calendar  approach  clearly  revealed 
any  gaps  in  the  desired  quality  criteria  of  diversity  in  teaching  models  and  styles;  diversity  in 
learning  styles,  achieving  the  upper  levels  of  Bloom’s  Taxonomy,  and  if  learning  was  active, 
rather  than  passive,  according  to  Dale’s  Cone  of  Learning. 


Course  Schedule  Reflecting  Teaching  Models  and  Styles;  Student  Learning  Styles; 
Bloom’s  Cognitive  Dimension;  and,  Dale’s  Active  Learning 

3 Faculty  Samples  follow  the  Legend  (Tables  B.8.I-3) 


1 


Teaching  Learning  Calendar/Content  Analysis  Legend 


Teaching  Models 
(Joyce,  Weil,  Calhoun,  2004) 


Teaching  Styles  Kolb’s  Learning  Styles 

(Mosston  & Ashworth,  1990)  (Kolb,  1984) 


IT  - inductive  thinking 

CA  - concept  attainment 

PWIM  - picture  word  induction  model 

Scl  - scientific  inquiry 

M - mneumonics 

S - synectics 

AO  - advance  organizers 
Partners 

CL-I  - cooperative  learning-informal 

CL-F  - cooperative  learning-formal 

SI  - structured  inquiry 

GI  - group  investigation 

RP  - role  playing 

JI  - jurisprudential  inquiry 

NT  - nondirective  teaching 

ES  - enhancing  self-esteem 

ML  - mastery  learning 

PS  - programmed  schedule 

DI  - direct  instruction 

S - simulation 


C - command  (A) 

P - practice  (B) 

R - reciprocal  (C) 

SC -self  check  (D) 

I - inclusion  (E) 

GD  - guided  discovery  (F) 

CD  - convergent  discovery  (G) 
DP  - divergent  production  (H) 
LD  - learner  designed  (I) 

LI  - learner  initiated  )J) 

ST -self  teach  (K) 


Teaching  Styles 
(Grasha,  1996) 

E - expert 
FA  - formal  authority 
PM  - personal  model 
F - facilitator 
D - delegator 


CE  - concrete  expeience 
RO  - reflective  observation 
AC  - abstract  conceptualization 
AE  - active  experimentation 


Bloom  (1956) 

K-R  - knowledge  or  remember 

C-U  - comprehension  or  understanding 

Ap  - application  or  apply 

An  - analyze 

S-E  - synthesize  or  evaluate 

E-C  - evaluate  or  create 

Dale’s  Cone  of  Learning 
(Dale,  1969) 

P - passive  (listening  only) 

I - intermediate 
(participating  in  discussion) 

A -active  (doing) 


2 


Table  B.8.1:  Faculty  Sample  1.  IENG370  - Moraga  VLCourse  Topics,  Class  Schedule  & Due 
Dates 


Week  & Objectives 

TM 

TS 

LS 

Day  1 Topics/Learning  Activities 
Due  Dates 

TM 

TS 

LS 

Day  2 Topics/Learning  Activities 
Due  Dates 

Week  8/28 

1 . Apply  OR  methods  to 
industrial  problems,  (1.1)  apply 
LP  models 

2.  Apply  LP  model  to  real-world 
problems,  (2.1)  choose  real- 
world  problem 

DI 

CS 

AC/ 

RO 

Introduction 

Individual  work:  formulation  case 
study  1 handed  out 

IT 

DI 

CDS 

AC/ 

RO 

Linear  Programming 

- assumptions 

- graphical  method 

Group  work:  formulation  case 
study  1 analyzed 

Week  9/4 

1 . Apply  OR  methods  to 
industrial  problems,  (1.1)  apply 
LP  models 

2.  Apply  LP  model  to  real-world 
problems,  (2.1)  choose  real- 
world  problem,  (2.2)  define  the 
problem  & (2.3)  define  the 
problem 

DI 

CS 

CE/ 

RO 

Linear  Programming 

- classical  examples 

- LP  formulation 

IT 

CDS 

AC/ 

RO 

Linear  Programming 
- LP  formulation 

Group  work:  formulation  case 
study  2 

Week  9/11 

1 . Apply  OR  methods  to 
industrial  problems,  (1.1)  apply 
LP  models 

2.  Apply  LP  model  to  real-world 
problems,  (2.4)  solve  the 
problem  using  SA 

DI 

CS 

CE/ 

RO 

Linear  Programming 

- standard  LP  form 

- reduction  techniques 

Individual  work:  rubric  on  simplex 
method  handed  out 

IT 

CDS 

AC/ 

RO 

Linear  Programming 

- LP  tabular  form 

- Simplex  Algorithm  fundamentals 

Group  work:  rubric  on  simplex 
method  analyzed 

Week  9/18 

1 . Apply  OR  methods  to 
industrial  problems,  (1.1)  apply 
LP  models 

2.  Apply  LP  model  to  real-world 
problems,  (2.4)  solve  the 
problem  using  SA 

Linear  Programming 
- SA  mechanic 

Linear  Programming 

- big  M 

- two  phase  method 

Week  9/25 

1 . Apply  OR  methods  to 
industrial  problems,  (1.1)  apply 
LP  models 

2.  Apply  LP  model  to  real-world 
problems,  (2.5)  solve  the 
problem  using  SA 

Duality  Theory  and  Sensitivity 
Analysis 

- economic  interpretation 

- primal-dual  relationship 

Duality  Theory  and  Sensitivity 
Analysis 

- dual-simplex  method 

- sensitivity  analysis 

Week  10/2 

1 . Apply  OR  methods  to 
industrial  problems,  (1.1)  apply 
LP  models 

2.  Apply  LP  model  to  real-world 
problems,  (2.5)  solve  the 
problem  using  SA 

Duality  Theory  and  Sensitivity 
Analysis 

- sensitivity  analysis 

Duality  Theory  and  Sensitivity 
Analysis 

- sensitivity  analysis 

- use  of  LINDO 

Week  10/9 

1 . Apply  OR  methods  to 
industrial  problems,  (1.1)  apply 
LP  models 

2.  Apply  LP  model  to  real-world 
problems 

Transportation  Problem 

- formulation  and  mathematical 
structure 

- applications 

Transportation  Problem 

- tabular  form 

- algorithm 

Week  10/16  - MIDTERM 

1 . Apply  OR  methods  to 
industrial  problems,  (1.1)  apply 
LP  models 

2.  Apply  LP  model  to  real-world 
problems 

Transportation  Problem 
- algorithm,  example 

Week  10/23 

1 . Apply  OR  methods  to 
industrial  problems,  (1.1)  apply 
LP  models 

2.  Apply  LP  model  to  real-world 
problems 

Assignment  Problem 
- formulation,  math  structure, 
algorithm  and  example 

Network  Flow  Models 

- terminology 

- shortest  path  problem 

3 


Week  10/30 

1 . Apply  OR  methods  to 
industrial  problems,  (1.1)  apply 
LP  models 

2.  Apply  LP  model  to  real-world 
problems 

1 

1 

Network  Flow  Models 

- maximum  spanning  tree 

- maximum  flow  problem 

1 

1 

Network  Flow  Models 

- maximum  flow  problem 

- minimum  cost  flow  problem 

Week  11/6 

1 . Apply  OR  methods  to 
industrial  problems,  (1.1)  apply 
LP  models  & (1.2)  apply  IP 
models 

1 

1 

Network  Flow  Models 
- minimum  cost  flow  problem 

1 

1 

Integer  Programming 

- assumptions  and  formulation 

- applications 

Week  11/13 
1 . Apply  OR  methods  to 
industrial  problems,  (1.2)  apply 
IP  models 

1 

1 

Integer  Programming 

- applications 

- Branch  and  Bound  algorithm 

1 

1 

Integer  Programming 

- Branch  and  Bound  algorithm 

- examples 

Week  11/20 
1 . Apply  OR  methods  to 
industrial  problems,  (1.3) 
understand  basic  knowledge  of 
DP 

1 

1 

Dynamic  Programming 
- characteristics  of  DP 

1 

1 

11/23  THANKSGIVING 

Week  11/27 
1 . Apply  OR  methods  to 
industrial  problems,  (1.3) 
acquire  basic  knowledge  of  DP 

1 

1 

Dynamic  Programming 
- steps  in  DP 

1 

1 

Dynamic  Programming 
-example 

Week  12/4 

2.  Apply  LP  model  to  real-world 
problems,  (2.6)  communicate 
results 

1 

1 

Project  presentations 

1 

1 

Project  presentations 

Week  12/11  FINAL  EXAM 


Table  B.8.2:  Faculty  Sample  2.  VI.  Course  Topics,  Class  Schedule  & Due  Dates Abul  Azad-Technology  277 


Week 

Obj 

TM 

TS 

LS 

B 

D 

Topics/Lab 
Activities  & Due 
Dates 

TM 

TS 

LS 

B 

D 

Topics/Lab  Activities 
& Due  Dates 

1 

8/28 

Inductive  thinking  / 
F 

Assimil 

ating 

Comprehe 

nsion 

Digital  Concepts 

Direct  instruction  and 
inductive  thinking/F 

Assimilating 

Application 

Receiving 

/participating 

Number  Systems 

2 

9/4 

Concept  attainment 
, and  Direct 
instruction  / B,  F 

Assimil 

ating 

Comprehe 

nsion 

Receiving  / 
participating 

Number  Systems 

Concept  attainment  and 
direct  instruction  / B,  F 

Assimilating 

Comprehensi 

on 

Receiving  / 
participating 

Logic  Gates 

3 

9/11 

Concept  attainment, 
and  Direct 
instruction  / A 

Assimil 

ating 

Applicatio 

n 

Receiving  / 
Participating 

Logic  Gates 

Concept  attainment  and 
direct  instruction  / B 

Assimilating 

Comprehensi 
on  / 

application 

Receiving  / 
Participating 

Logic  Gates 

4 

9/18 

Inductive  thinking  / 
B,D 

Assimil 

ating 

Applicatio 

n 

Receiving  / 
Participating 

Logic  Gates 

Direct  instruction. 
Content  attainment,  and 
Partners  in  learning  / A, 
B,H 

Assimilating 
/ Diverging 

Comprehensi 

on 

Receiving  / 
Participating 

Boolean  Algebra  and  Logic 
Simplification 

Performance  Test-1 

5 

9/25 

Content  attainment, 
and  Inductive 
thinking  / B,  D 

Assimil 

ation 

Applicatio 

n 

Receiving  / 
Participating 

Boolean  Algebra  and  Logic 
Simplification 

Direct  instruction,  and 
Content  attainment  / A,  B 

Assimilating 

Comprehensi 

on 

Receiving  / 
Participating 

Combinational  Logic 
Analysis 

6 

10/2 

Inductive  thinking, 
and  Content 
attainment  / B,  D 

Assimil 

ation 

Applicatio 

n 

Receiving  / 
Participating 

Combinational  Logic 
Analysis 

Inductive  thinking,  and 
Content  attainment  / B,  D 

Assimilation 

Application 

Doing 

Combinational  Logic 
Analysis 

7 

10/9 

Direct  instruction, 
and  Inductive 
thinking  / A,  B 

Assimil 

ating 

Comprehe 

nsion 

Visual  receiving 

Functions  of 
Combinational  Logic 

Inductive  thinking,  and 
Content  attainment  / B,  D 

Assimilation 

Application 

Receiving  / 
Participating 

Functions  of  Combinational 
Logic 

8-10/16 

MT 

Mnemonics, 
Inductive  thinking, 
and  Content 
attainment  / B,  D 

Assimil 

ation 

Applicatio 

n 

Receiving  / 
Participating 

Functions  of 
Combinational  Logic 

MIDTERM 

9 

10/23 

Inductive  thinking  / 
B,D 

Assimil 

ation 

Applicatio 

n 

Doing 

Functions  of 
Combinational  Logic 

Inductive  thinking. 
Partners  in  learning. 
Group  investigation  / B, 
D,H 

Assimilation 
/ Diverging 

Application 

Doing 

Functions  of  Combinational 
Logic 

Performance  Test-2 

10 

10/30 

Direct  instruction, 
and  Inductive 
thinking  / A,  B 

Assimil 

ating 

Comprehe 

nsion 

Visual  receiving 

Latches,  Flip-Flops,  and 
Timers 

Content  attainment  and 
Inductive  thinking  / B,  D 

Assimilation 

Application 

Receiving  and 
participating 

Latches,  Flip-Flops,  and 
Timers 

5 


11 

11/6 

Content  attainment 
and  Indnctive 
thinking  / B,  D 

Assimil 

ation 

Applicatio 

n 

Receiving  and 
participating 

Latches,  Flip-Flops,  and 
Timers 

Direct  instruction  and 
Content  attainment  / A,  B 

Assimilation 

Comprehensi 

on 

Visual 

Receiving  and 
Participating 

Counters 

■ 

Indnctive  thinking 
and  Content 
attainment  / B,  D 

Assimil 

ation 

Comprehe 

nsion 

Receiving  and 
Participating 

Counters 

Inductive  thinking  and 
Content  attainment  / B,  D 

Assimilation 

Application 

Visual 

Receiving  and 
Participating 

Counters 

Direct  instruction 
and  Content 
attainment  / A,  B 

Assimil 

ation 

Comprehe 

nsion 

Visual 

Receiving  and 
Participating 

Shift  Registers 

11/23  THANKSGIVING 

14 

11/27 

Inductive  thinking 
and  Content 
attainment  / B,  D 

Assimil 

ation 

Applicatio 

n 

Receiving  and 
Participating 

Shift  Registers 

Inductive  thinking  and 
Content  attainment  / B,  D 

Assimilation 

Application 

Visual 

Receiving  and 
Participating 

Shift  Registers 

Inductive  thinking 
and  Content 
attainment  / B,  D 

Assimil 

ation 

Applicatio 

n 

Receiving  and 
Participating 

Shift  Registers 

Inductive  thinking, 
Partners  in  learning, 
Group  investigation  / B, 
D,H 

Assimilation 
/ Diverging 

Application 

Doing 

Performance  Test-3 

16-2/11 

FE 

Inductive  thinking 
and  Content 
attainment  / B,  D 

Assimil 

ation 

Applicatio 

n 

Receiving  and 
Participating 

Shift  Registers 

Final  Examination 

6 


Table  B.8.3:  Faculty  Sample  3.  VI.  Course  Topics,  Class  Schedule  & Due  Dates 


lENG  475-  Decision  Analysis  - Regina  Rahn 


Week  Obj 

TS/TM 

LS 

Bloom/ 

Dale 

Topics/Lab  Activities 
& Due  Dates 

TS/TM 

LS 

Bloom/ 

Dale 

Topics/Lab  Activities 
& Due  Dates 

1 

8/28 

Students  will  be  able  to 
constnact/create  a 
decision  tree  to  aid  in 
determining  the  best 
course  of  action  for  a 
given  set  of 
circumstances 

c 

E 

/ 

DI 

AC 

KR 

/ 

P 

Introduction 

Decision  Analysis 

Decision 

Trees 

Structure  and 
Outcomes 

C 

E 

/ 

DI 

AC 

KR, 

CU 

/ 

P 

Decision  Trees 

Probability  Overview 

Decision  Trees 

Expected  V alue  and 
EMV 

2 

9/4 

Students  will  be  able  to 
construct/create  a 
decision  tree  to  aid  in 
determining  the  best 
course  of  action  for  a 
given  set  of 
circumstances 

Students  will  be  able  to 
perform  sensitivity 
analysis  on  a decision 
tree  scenario 

c 

E 

/ 

DI,  AO 

AC 

KR, 

CU 

/ 

P 

Decision  Trees 

EVPI  and  EVSI 

Decision  Trees 

Probability 
Calculations  - Baye’s 
Theorem 

C,P 

E,  EA 
/ 

DI,  AO,  IT, 

SI, 

DEDUCTIVE 

AE, 

CE, 

AC, 

RO 

KR, 

CU, 

AP 

/ 

I 

Decision  Trees 

Sensitivity  Analysis 

Decision  Trees 

Case  Study 

HW#lDue 

(Students  will  be  able  to 
construct/create  a decision 
tree  to  aid  in  determining 
the  best  course  of  action  for 
a given  set  of 
circumstances) 

3 

9/11 

Students  will  be  able  to 
perform  sensitivity 
analysis  on  a decision 
tree  scenario 

C,P 

E,  EA 
/ 

DI,  AO,  IT, 

SI, 

DEDUCTIVE 

CE, 

AC, 

RO 

KR, 

CU, 

A 

/ 

I 

Decision  Trees 

Case  Study  Continued 

R,  SC,  DP 

E 

/ 

IT,  SI,  CL-I 

CE, 

AE, 

AC, 

RO 

AN, 

AP, 

SE 

/ 

A 

Small  Group 
Encounter 

(Students  will  be  able  to 
define  a scenario  to  be 
analyzed  with  the  decision 
analysis  techniques 

Students  will  be  able  to 
formulate/design  a possible 
solution  approach  to  the 
given  problem) 

7 


4 

9/18 

Students  will  be  able  to 
constnact  the  formulae 
for  conditional 
likelihood  ratios  and  to 
calculate  the 
probabilities/ratios 

Students  will  be  able  to 
make  inferences  about 
a system  based  on  the 
values  of  the 
conditional  likelihood 
ratios 

C,P 

E,  FA 
/ 

DI,  SI 

AE, 

AC 

KR, 

CU, 

AP 

/ 

P 

Decision  Trees 
Conditional 
Likelihood  Ratios 

HW#2Due 

(Students  will  be  able  to 
construct/create  a decision 
tree  to  aid  in  determining 
the  best  course  of  action  for 
a given  set  of  circumstances 

Students  will  be  able  to 
perform  sensitivity  analysis 
on  a decision  tree  scenario) 

C,  I,  GD,  CD, 
DP, 

E,  FA,  D 
/ 

DI,  SI,  IT 

AE, 

AC 

KR, 

CU, 

AP, 

AN, 

SE, 

EC 

/ 

P 

Utility  Theory 

Introduction 

Motivation  and 
Axioms 

Performance  Task  #1 
- written  report  due 

(To  demonstrate  an  ability 
to  effectively  present  the 
problem,  solution,  and 
recommendations  of  a 
complicated  decision 
scenario  - a written  report) 

5 

9/25 

Students  will  be  able  to 
constnact  a single- 
attribute utility 
function 

C 

E 

/ 

DI 

AC 

KR, 

CU 

/ 

P 

utility  Theory 

Degrees  of  Risk 

R,  I,  CD,  DP 

F,D 

/ 

IT,  SI 

CE, 

AE, 

AC, 

RO 

/ 

A 

Performance  Task  #1 
- oral  presentations 

(To  demonstrate  an  ability 
to  effectively  present  the 
problem,  solution,  and 
recommendations  of  a 
complicated  decision 
scenario  - a presentation) 

6 

10/2 

Students  will  be  able  to 
evaluate  a given  utility 
function 

To  structure  (and 
solve)  complicated 
decision  problems 

C,P 

E,  FA 
/ 

DI,  AO 

CE, 

AC, 

RO 

KR, 

CU, 

AP 

/ 

I 

Utility  Theory 

Lotteries 

Lotteries  - 
Calculations  and 
Examples 

R,  I,  CD,  DP, 
ED 

F,D 

/ 

DI,  IT,  SI, 
CE-F,  STI,  GI 

AE, 

AC, 

RO 

KR, 

CU, 

AP, 

AN, 

SE 

/ 

P 

utility  Theory 

Two  Lotteries  at  a 
Time 

Single  Attribute 
Utility  Functions 

Performance  Task  #2 
- Phase  1:  Project 
Selection  Due 

(Students  will  be  able  to 
define  a scenario  to  be 
analyzed  with  the  decision 
analysis  techniques) 

8 


7 

10/9 

To  structure  (and 
solve)  complicated 
decision  problems 


p,  R,  CD 

FA,  F 
/ 


CE, 

AN, 

AE, 

SE 

AC, 

/ 

RO 

A 

IT,  SI,  CL-I, 
STI,  GI 


Class 

Discussion: 
Topic  - 
Project 
Selection 


P,  R,  CD 

FA,  F 
/ 

AO,  IT,  SI 


CE, 

AN, 

AE, 

SE 

AC, 

/ 

RO 

A 

Interactive 

Review 

Session 


HI/I/ #3  Due 

(Students  will  be  able  to 
construct  a single-attribute 
utility  function 


8-10/16  MT 

To  leam  to  use  a 
specific  set  of 
analytical  tools  for 
technical  decision 
making  under 
uncertainty. 


p 

FA 

/ 

IT,  SI, 


AE,  CU, 
AC  AP, 
AN 
/ 

A 


Students  will  be  able  to 
evaluate  a given  utility 
function) 


Midterm 

Exam 


c 

E 

/ 

DI 


AC,  KR 
RO  / 

I 


Fault 


(Discuss 


To  apply  analytic 
decision  making 
techniques  for 
technical  decision 
making  under 
uncertainty,  and  to 
analyze  and  evaluate 
the  results. 


DEDUCTIVE 


Trees 


Midterm) 


To  Structure  and  solve 
complicated  decision 
problems 

To  identify  and  define 
any  limitations  of  the 
models  and  techniques 
for  rational  decision- 
making. 


9 


9 

10/23 

Students  will  be  able  to 
develop  and  analyze 
fault  trees 


c 

AC, 

KR 

c 

AC 

KR 

RO 

/ 

Fault  Trees 

/ 

Fault  Trees 

E 

P 

Evaluation  - 

E 

P 

Evaluation  - 

/ 

Qualitative 

/ 

Quantitative 

DI 

DI 

C,P 

CE, 

KR, 

R,  I,  CD,  DP, 

CE, 

AN, 

Round  Table 

AC, 

CU, 

Fault  Trees 

ED 

AE, 

SE, 

Discussion 

E,  EA 

RO 

AP 

Case  Study 

AC, 

EC 

Performance  Task  #2 

/ 

/ 

F,D 

RO 

/ 

- Phase  2:  Formulate 

DI,  AO,  IT, 

I 

Dual  Fault  Trees 

/ 

A 

and  Measure  Due 

SI 

IT,  SI,  CE-I, 

(Students  will  be  able  to 
formulate/design  a possible 

CE-P,  STI,  GI 

solution  approach  to  the 
given  problem 

Students  will  be  able  to 
implement  the  solution 
techniques  to  obtain  a first 
round  solution) 

C 

AC 

KR 

Multi-Attribute 

C,P 

CE, 

KR, 

Multi-Attribute 

/ 

Utility  Theory 

AC, 

CU, 

Utility  Theory 

E 

p 

(MAU) 

E,  PA 

RO 

AP 

(MAU) 

/ 

The  General  Case 

Additive  Functions 

/ 

/ 

Calculations  and 

DI 

DI 

I 

Examples 

P,  R,  CD 

CE, 

AP, 

R,  I,  CD 

CE, 

AP, 

AE, 

AN 

Homework 

discussion 

AE, 

AN 

Round  Table 

PA,  E 

AC, 

/ 

PA,  P 

AC, 

/ 

Discussion  of 

/ 

RO 

A 

HW#4Due 

/ 

RO 

A 

Handouts  on 

AO,  IT,  SI, 

Students  will  be  able  to 

AO, 

MAU 

CE-I 

develop  and  analyze  fault 

DEDUCTIVE, 

trees 

SI,  CE-I 

(Students  will  be  able  to 
evaluate  a given  utility 

Students  will  be  able  to 
evaluate  the  scenarios 
modeled  by  a fault  tree) 

function) 

Possible  guest  lecturer 

10 

10/30 

Students  will  be  able  to 
evaluate  the  scenarios 
modeled  by  a fault  tree 


11 

11/6 

Students  will  be  able  to 
construct  a multi- 
attribute utility 
function 

Students  will  be  able  to 
implement  the  additive 
model  for  multi- 
attribute  utility  theory 


12 

11/13 

Students  will  be  able  to 
develop  and  analyze 
fault  trees 

Students  will  be  able  to 
evaluate  the  scenarios 
modeled  by  a fault  tree 

Students  will  be  able  to 
evaluate  a given  utility 
function 


10 


13 

11/20 

Students  will  be  able  to 
construct  a multi- 
attribute utility 
function 

Students  will  be  able  to 
implement  the  additive 
model  for  multi- 
attribute utility  theory 

C,  R,  I,  CD, 
DP,  LD 

E,  F,D 
/ 

DI,  IT,  SI, 
CL-F,  STI, 
GI, 

AE, 

AC, 

RO 

KR, 

SE, 

EC 

/ 

P 

Multi-Attribute 
Utility  Theory 
(MAU) 

More  Calculations  and 
Examples 

Performance  Task  #3 
- Phase  3:  Analyze 
Summary  Due 

(Students  will  be  able  to 
analyze  the  solution  found 
in  order  to  assess  the  cuiTent 
state) 

11/23  Thanksgiving 

14 

11/27 

Students  will  be  able  to 
construct  a multi- 
attribute utility 
function 

Students  will  be  able  to 
implement  the  additive 
model  for  multi- 
attribute utility  theory 

P,  R,  CD 

FA,  F 
/ 

AO,  IT,  SI, 
CL-I 

CE, 

AE, 

AC, 

RO 

AP, 

AN 

/ 

A 

Homework 

discussion 

HW  #5  Due 

(Students  will  be  able  to 
construct  a multi-attribute 
utility  function 

Students  will  be  able  to 
implement  the  additive 
model  for  multi-attribute 
utility  theory) 

Possible  guest 
lecturer 

P,  R,  CD 

PA,  E 
/ 

AO,  IT,  SI 

CE, 

AE, 

AC, 

RO 

AP, 

AN 

/ 

A 

Interactive  Review  of 
Course 

11 


15 

R,  I,  CD,  DP, 

CE, 

KR, 

R,  I,  CD,  DP, 

CE, 

KR, 

12/4 

To  demonstrate  an 

LD,LI 

AE, 

CU, 

Performance 

ED,  El 

AE, 

CU, 

Project 

ability  to  effectively 
present  the  problem. 

F,D 

AC, 

AP, 

Task  #3  - 

E,D 

AC, 

AP, 

Presentations 

solution,  and 

RO 

AN, 

Phase  4: 

RO 

AN, 

(To  demonstrate  an  ability 

recommendations  of  a 
complicated  decision 
scenario 

/ 

IT,  SI,  CL-I, 

SE, 

EC 

Improve  and 
Control  must 

/ 

IT,  SI,  CE-I, 

SE, 

EC 

to  effectively  present  the 
problem,  solution,  and 
recommendations  of  a 

To  identify  and  define 
any  limitations  of  the 

CL-F,  STI,  GI 

/ 

CE-E,  STI,  GI 

/ 

complicated  decision 

A 

be  completed 

A 

scenario) 

models  and  techniques 

(Students  will  be  able  to 

for  rational  decision- 

recommend  a course  of 

making 

action  based  on  the  original 
solution 

To  structure  and  solve 
complicated  decision 

Students  will  be  able  to 

problems 

assess  the  usefulness  of  the 
solution  and  recommend 
any  changes  in  the  process 

To  identify  and  define  any 
limitations  of  the  models 
and  techniques  for  rational 
decision-making.) 

Project 

Presentations 

(To  demonstrate  an  ability 
to  effectively  present  the 
problem,  solution,  and 
recommendations  of  a 
complicated  decision 
scenario) 

12 


16-2/11 

FE 

To  learn  to  use  a 
specific  set  of 
analytical  tools  for 
technical  decision 
making  under 
uncertainty. 


p 

FA 

/ 

IT,  SI, 

DEDUCTIVE 


AE, 

AC, 

RO 


KR, 

AP, 

AN 

/ 

A 


To  apply  analytic 
decision  making 
techniques  for 
technical  decision 
making  under 
uncertainty,  and  to 
analyze  and 
evaluate  the 
results. 


Final  Exam 

(To  learn  to  use  a specific 
set  of  analytical  tools  for 
technical  decision  making 
under  uncertainty. 

To  apply  analytic  decision 
making  techniques  for 
technical  decision  making 
under  uncertainty,  and  to 
analyze  and  evaluate  the 
results.) 


To  demonstrate  an 
ability  to 

effectively  present 
the  problem, 
solution,  and 
recommendations 
of  a complicated 
decision  scenario 


To  identify  and 
define  any 
limitations  of  the 
models  and 
techniques  for 
rational  decision- 
making 

To  structure  and 
solve  complicated 
decision  problems 
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R,  I,  CD,  DP, 

AE, 

KR, 

ED,  El 

AC, 

CU, 

RO 

AP, 

F,D 

AN, 

/ 

SE, 

IT,  SI,  CE-I, 

EC 

CE-E,  STI,  GI 

/ 

A 

Performance  Task  #3  - 
Final  Written  Reports 
Due 

(To  demonstrate  an  ability  to 
effectively  present  the  problem, 
solution,  and  recommendations 
of  a complicated  decision 
scenario 

To  identify  and  define  any 
limitations  of  the  models  and 
techniques  for  rational  decision- 
making 

To  structure  and  solve 
complicated  decision  problems) 


DISCUSSION  OF  PROFESSORS’  2005  AND  2006  TEST  AND  TEST  ANALYSIS 

COMPARISONS 
Jerry  Gilmer,  Ph.D. 

Each  professor  submitted  copies  of  the  midterm  and  final  exams  they  used  in  their  classes  during 
the  Fall  2005  semester  and  the  corresponding  exams  they  used  during  the  Fall  2006  semester. 

The  Fall  2006  exams  were  developed  during  and  after  formal  training  in  test  development;  it  is 
expected  that  the  2006  exams  would  be  improved  over  the  2005  exams.  The  professors  also 
learned  formal  item  analysis  procedures  and  generated  item  analyses  for  their  exams  from  both 
Fall  2005  and  Fall  2006  semesters. 

The  professors’  exams  and  the  item  analysis  results  were  compared  and  some  general  comments 
from  the  comparisons  are  included  below.  During  the  test  development  training,  professors 
learned  about  different  characteristics  of  good  exams,  and  it  is  these  characteristics  that  were 
used  for  making  comparisons  from  2005  to  2006.  Characteristics  of  item  analysis  were  also 
presented  to  the  professors  and  considered  for  comparisons. 

The  discussion  below  includes  a brief  statement  regarding  the  meaning  of  each  characteristic  and 
then  a brief  evaluation  related  to  the  changes  in  tests  and  item  analyses  from  2005  to  2006.  The 
characteristics  were  applied  for  evaluation  to  each  of  the  exams  and  analyses  submitted  by  each 
professor. 

Test  Characteristics 

Based  on  Item  Bank  - Is  the  exam  based  on  an  item  bank  - a pool  of  items?  During  the  test 
development  sessions  professors  were  asked  to  develop  an  item  bank  covering  all  of  the  content 
areas  in  their  course. 

Evaluation  - It  appears  that  none  of  the  exams  for  2005  were  based  on  an  item  bank  and  that 
all  of  the  exams  for  2006  were  based  on  an  item  bank. 

Based  on  Table  of  Specifications  - Is  the  exam  based  on  a formal  plan,  a blueprint,  also  called  a 
table  of  specifications?  The  professors  were  taught  how  to  create  and  utilize  a table  of 
specifications  to  ensure  the  test  covers  all  the  intended  material  in  appropriate/specified 
proportions. 

Evaluation  - It  appears  that  none  of  the  exams  for  2005  were  based  on  a table  of 

specifications  and  that  all  of  the  exams  for  2006  were  based  on  a table  of  specifications. 

Overall  Appearance  - well  laid  out;  pleasing  appearance,  grammar,  etc.  - The  exam  should 
be  generally  pleasing  to  look  at  and  easy  to  read  and  follow.  Characteristics  such  as  font 
selection,  spacing,  use  of  highlights  such  as  indents,  bold  letters,  etc.  should  be  considered. 

Evaluation  - The  2006  exams  were  much  improved  over  the  2005  exams  for  six  of  the  seven 
professors.  In  general  appearance,  the  2006  exams  from  one  professor  looked  similar  to 
the  2005  exams. 

Overall  Instructions  - clear,  unambiguous  - The  exam  should  include  some  overall 
instructions  to  the  students,  and  these  instruction  should  appear  on  the  first  page  of  the  exam. 
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Evaluation  - Four  professors  included  very  good  and  improved  instructions  for  their  2006 
exams.  Two  instructors  did  not  provide  any  overall  instructions,  and  one  professor 
modified  the  instructions  only  slightly. 

Instructions  for  Item  Subsets  - clear  unambiguous  - Sometimes  instructions  are  necessary  for 
item  sets  - a group  of  multiple  choice  items,  or  a group  of  short-answer  items,  for  example,  or  a 
set  of  items  related  to  a common  diagram  or  a common  reading  passage. 

Evaluation  - Four  of  the  professors  included  item  sets,  and  their  instructions  for  these  sets 
for  2006  were  generally  very  good.  Three  of  the  professors  did  not  use  item  sets. 

Number  of  short,  discrete  items  vs.  longer  items  - The  test  development  sessions  covered  the 
use  of  more  short,  discrete  items  rather  than  fewer  longer  items  to  ensure  that  the  exam  provides 
an  adequate  sampling  of  the  course  material. 

Evaluation  - All  of  the  2006  exams  were  improved  on  this  characteristic.  Most  professors 
used  long,  problem-type  items  for  their  2005  exams  but  converted  to  more  discrete  items 
for  their  2006  exams.  Two  professors  added  only  a small  number  of  discrete  items  and 
still  used  few,  problem-type  items. 

Number  of  objectively  scored  vs.  subjectively  scored  items  - Although  subjectively  scored 
items  (short-  or  long-answer,  problem-type,  requiring  scorer  judgments)  are  sometimes 
appropriate,  the  use  of  objectively  scored  items  (multiple  choice,  true-false,  etc.)  is  often 
preferred.  They  are  easier  and  more  efficient  to  score,  and  generally  yield  a more  reliable  exam. 

Evaluation  - All  professors,  along  with  using  more  short,  discrete,  items,  used  items  that 
could  be  objectively  scored.  Most  professors  included  a few  subjectively  scored  items  as 
well.  Two  professors  used  only  subjectively  scored  items  on  the  final  exam;  one  of  those 
did  so  after  discussing  the  midterm  with  the  class  and  determining  that  subjectively 
scored  items  were  more  appropriate  to  the  course  content. 

Item  Quality  - clear,  direct,  well  written,  no  clues  - The  items  should  be  well- written,  using 
appropriate  English,  and  should  contain  no  clues  or  cues. 

Evaluation  - This  characteristic  is  difficult  for  a non-expert  in  the  content  to  judge,  but 
generally  the  exams  appeared  to  be  clear  and  understandable.  For  two  professors,  the 
2006  exams  seemed  to  exhibit  no  changes.  For  the  other  five  professors,  the  2006  exams 
exhibited  noticeable  improvement.  But  the  2005  exams  appeared  to  be  satisfactory  to 
begin  with. 

Item  Analysis  Characteristics 

Item  Difficulty  - number  of  very  difficult  items  - In  general,  the  items  should  not  be  too 

difficult.  If  less  that  50%  of  possible  points  for  an  item  were  awarded,  the  item  was  judged  to  be 
too  difficult.  This  threshold  (50%)  is  not  inappropriate  and  was  used  consistently  to  evaluate  the 
exams  and  item  analyses  submitted  by  the  professors.  But  the  professors  were  informed  that  they 
should  determine  their  own  threshold,  one  that  is  meaningful  to  them  and  their  exams  and  that 
there  is  no  specific  value  that  should  be  used  forjudging  items  in  all  situations. 

Evaluation  - Some  professors’  2005  exams  contained  a small  number  of  problems,  and 
these  problems  had  generally  high  (easy)  difficulty  indices.  All  the  analyses  for  the  2006 
exam  analyzed  several  more  items.  It  appeared  that  more  items  were  flagged  as  being  too 


13 


difficult  (difficulty  index  less  than  50%)  than  would  normally  be  desired.  New  item  types 
were  being  used  for  the  first  time  by  most  of  the  professors,  and  they  are  still  learning 
how  to  use  item  analysis  data  to  improve  item  performance  for  future  administrations. 


Item  Discrimination  - number  of  poorly  discriminating  items  - In  general  the  items  should 
discriminate  between  students  proficient  in  the  content  and  students  who  are  less  proficient. 
Items  were  flagged  if  the  discrimination  index  was  less  than  0.15.  This  threshold  is  not 
completely  arbitrary,  but  as  with  the  difficulty  index,  professors  were  counseled  to  choose  a 
threshold  that  is  meaningful  and  appropriate  for  their  context.  Professors  were  also  informed, 
however,  that  an  item  should  not  have  a negative  discrimination  index. 

Evaluation  - For  the  2006  exams,  there  were  probably  more  poorly  discriminating  and 
negative  discriminating  items  than  would  be  desired.  As  indicated  above,  however,  new 
item  types  were  being  used  for  the  first  time  by  most  of  the  professors,  and  they  are  still 
learning  how  to  use  item  analysis  data  to  improve  item  performance  for  future 
administrations. 


14 


Table  B.9.a.l:  2005  vs.  2006  Test  and  Test  Analysis  Comparisons  - Ibrahim  Abdel-Motaleb 


Midterm 

Final 

Test  Comparisons 

Fall  2005 

Fall  2006 

Fall  2005 

Fall  2006 

Based  on  Item  Bank 

No 

Yes 

No 

Based  on  Table  of 
Specifications 

No 

Yes 

No 

Overall  Appearance  - well 
laid  out;  pleasing 
appearance,  grammar, 
etc. 

OK.  Test  is  only 
one  item,  3 parts, 
20  pts. 

Very  good!  33 
Items. 

Take  home  exam. 
One  item,  3 parts, 
50  pts. 

Overall  Instructions  - 
clear,  unambiguous 

Not  reaiiy  there. 
Only  one  item. 

Not  much  there. 

Instructions  may  be 
satisfactory.  But  pt 
values  for  items  & 
overall  test  not 
specified. 

Instructions  for  Item 
Subsets  - clear, 
unambiguous 

NA 

Yes.  Some  for 
matching  and 
others. 

NA 

Number  of  short,  discrete 
items  vs  longer  items 

One  long  item  - 20 
pts. 

Test  contains 
mostly  short, 
discrete  items 
rather  than  a single 
long  item. 

One  long  item,  3 
parts,  50  pts. 

Number  of  objectively 
scored  vs  subjectively 
scored  items 

Item  is  subjectively 
scored.  No  scoring 
criteria  provided  to 
students. 

Several  MC  items 
(objectively  scored) 
and  short  answer 
items  (subjectively 
scored). 

Subjectively 
scored.  No  scoring 
criteria  provided. 

Item  Quality  - clear, 
direct,  well  written,  no 
clues 

Appears  OK. 

Appears  OK. 

Appears  OK. 

Item  Analysis 
Comparisons 

One  item,  3 parts, 
20  pts. 

33  items  but  only 
20  in  item  analysis 
(?) 

One  item,  3 parts, 
50  pts 

Item  Difficulty  - number 
of  very  difficult  items 

diff  - 67%,  the  3 
parts  not  analyzed 
separately. 

Diff:  16  items  > 
50% 

4 items  < 50% 

Lowest  diff  = 72%. 
Overall  test 
average  = 77% 

Item  Discrimination  - 
number  of  poorly 
discriminating  items 

Can't  compute 
since  analyzed  as 
one  item,  not  for 
three  parts. 

Disc:  2 items  0.0- 
.15 

1 item  negative. 

Lowest  disc  = .69. 
But  high  disc 
expected  since 
only  3 parts. 
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Table  B.9.a.2:  2005  vs.  2006  Test  and  Test  Analysis  Comparisons  - Abul  Azad 


Midterm 

Final 

Test  Comparisons 

Fall  2005 

Fall  2006 

Fall  2005 

Fall  2006 

Based  on  Item  Bank 

No 

Yes 

No 

Yes 

Based  on  Table  of 
Specifications 

No 

Yes 

No 

Yes 

Overall  Appearance  - well 
laid  out;  pleasing 
appearance,  grammar, 
etc. 

Generally  good. 
Tests  called  First 
In-course  Exam 
and  Second  In- 
course  Exam. 
Total  of  18  Items. 

About  the  same  as 
F05;  called 
Midterm. 

Total  of  12  Items. 

Generally  good. 
Only  8 Items. 

Similar  to  F05. 

7 Items  (last  Item 
mislabeled  as  #8) 
10  parts:  la,  1b, 
1c,  etc. 

Overall  Instructions  - 
clear,  unambiguous 

No  overall  test 
Instructions. 

No  overall  test 
Instructions. 

No  overall  test 
Instructions. 

No  overall  test 
Instructions. 

Instructions  for  Item 
Subsets  - clear, 
unambiguous 

NA 

NA 

NA 

NA 

Number  of  short,  discrete 
items  vs  longer  items 

Combination  of 
short-answer  and 
longer-answer 
Items.  But  Items 
have  multiple  parts 
(la,  1b,  1c,  etc. 

Fewer  short-  and 
long-answer  Items. 
Five  MC  Items. 

Most  Items  appear 
to  be  longer, 
problem-type; 
remainder  are 
short-answer. 
Some  with  multiple 
parts  (la,  1b,  1c). 

All  Items  are  short- 
er long-answer 
type 

Number  of  objectively 
scored  vs  subjectively 
scored  items 

All  Items  appear 
subjectively 
scored. 

Five  of  the  twelve 
Items  are 

objectively  scored. 

All  are  subjectively 
scored. 

All  are  subjectively 
scored. 

Item  Quality  - clear, 
direct,  well  written,  no 
clues 

Items  appear  clear 
and  direct. 

Items  appear  clear 
and  direct. 

Items  appear  clear 
and  direct. 

Items  appear  clear 
and  direct. 

Item  Analysis 
Comparisons 

First  2 tests  - 18 
Items  = 200  pts;  12 
Items  & 6 Items 

12  Items  = 100  pts. 

8 Items  - 100  pts. 

1 0 Items/parts 
analyzed  100  pts. 

Item  Difficulty  - number 
of  very  difficult  items 

14  of  18  Items 
analyzed. 

All  Items  were  very 
easy:  smallest  diff 
= 72%. 

1 dIff  = 20% 

Very  easy  - 
smallest  diff  = 70%. 

Lowest  diff  = 54% 
Others  61  %-81% 

Item  Discrimination  - 
number  of  poorly 
discriminating  items 

3 Items  had 
negative 
discrimination 
Indices  - partly 
because  they  were 
so  easy  and  high 
scorers  got  them 
wrong. 

4 Items  disc  In  0.0- 
.15. 

1 Item  negative 
disc. 

2 Items  with  disc 
<.10 

No  negative 
discriminators. 

Lowest  disc  = .44 
Others  .71 -.94 
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Table  B.9.a.3:  2005  vs.  2006  Test  and  Test  Analysis  Comparisons  - Brianno  Coder 


Midterm 

Final 

Test  Comparisons 

Fall  2005 

Fall  2006 

Fall  2005 

Fall  2006 

Based  on  Item  Bank 

No 

Yes 

No 

Yes 

Based  on  Table  of 
Specifications 

No 

Yes 

No 

Yes 

Overall  Appearance  - well 
laid  out;  pleasing 
appearance,  grammar, 
etc. 

Not  really  a 
midterm;  just 
quizzes  #3,  #4  and 
#5.  15  pts  and  2 
problems  each 
quiz.  Total  45  pts. 

Great  looking 
exam! 

34  Items,  36  pts. 
Pts.  specified  for 
each  Item. 

Doesn't  really  have 
the  formal 
appearance  of  an 
exam.  4 problems, 
1 5 pts  each  = 60 
pts. 

Great  looking 
exam! 

33  Items,  33  pts. 
Pts.  specified  for 
each  Item. 

Overall  Instructions  - 
clear,  unambiguous 

NA 

Great  - 

Comprehensive 

None 

Good 

Instructions  for  Item 
Subsets  - clear, 
unambiguous 

NA 

Very  clear 

NA 

Very  clear 

Number  of  short,  discrete 
items  vs  longer  items 

4 short-answer 
problems,  2 MC 

All  are  short, 
discrete  Items 

4 long  problems. 

All  are  short, 
discrete  Items 

Number  of  objectively 
scored  vs  subjectively 
scored  items 

Only  2 Items 
objectively  scored. 

All  are  objectively 
scored 

All  subjectively 
scored. 

All  are  objectively 
scored 

Item  Quality  - clear, 
direct,  well  written,  no 
clues 

OK 

Very  good 

OK 

Very  good 

Item  Analysis 
Comparisons 

Each  quiz  analyzed 
separately  - not 
combined,  2 
problems  each. 

34  Items  analyzed 

4 problems 
analyzed 

33  Items  analyzed 

Item  Difficulty  - number 
of  very  difficult  items 

1 (of  6)  diff  <50% 
Oulz  means  are 
61%,  52%  and 
42% 

14  Items  dIff  < 50% 
Test  mean  = 55% 

1 problem  diff 
=50% 

Test  mean  = 58% 

9 Items  diff  < 50% 
Test  mean  = 59% 

Item  Discrimination  - 
number  of  poorly 
discriminating  items 

Al  6 disc  >.56 

2 Items  disc  0.0-.15 

AII4dlsc>.58 

3 Items  disc  0.0-.15 
1 Item  disc 
negative 
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Table  B.9.a.4:  2005  vs.  2006  Test  and  Test  Analysis  Comparisons  - Abhijit  Gupta 


Midterm 

Final 

Test  Comparisons 

Fall  2005 

Fall  2006 

Fall  2005 

Fall  2006 

Based  on  Item  Bank 

No 

Yes 

No 

Yes 

Based  on  Table  of 
Specifications 

No 

Yes 

No 

Yes 

Overall  Appearance  - well 
laid  out;  pleasing 
appearance,  grammar, 
etc. 

Not  much  here.  3 
problems.  Slightly 
difficult  to  read. 
Poor  photocopy? 

A much  better- 
looking exam  than 
F05. 

Could  be  better. 

Only  4 items. 
Poor  photocopy. 

Much  improved 
appearance  over 
F05. 

Overall  Instructions  - 
clear,  unambiguous 

OK 

Good.  Includes  pt. 
values  for  each 
item. 

OK 

Good,  with  pt 
values  for  each 
item. 

Instructions  for  Item 
Subsets  - clear, 
unambiguous 

NA 

Yes,  good. 

NA 

Very  good! 

Number  of  short,  discrete 
items  vs  longer  items 

All  3 items  are  long 
problem  type. 

All  but  3 items  are 
MC,  TF. 

41  total  items. 

All  4 are  long 
problem  type. 

All  but  4 items  are 
MC,  TF. 

30  total  items. 

Number  of  objectively 
scored  vs  subjectively 
scored  items 

All  subjectively 
scored. 

All  but  3 are 
objectively  scored. 

All  subjectively 
scored. 

26  items  are 
objectively  scored. 

Item  Quality  - clear, 
direct,  well  written,  no 
clues 

Probably  OK. 

Very  good. 

OK 

Very  good! 

Item  Analysis 
Comparisons 

3 items 

41  items  analyzed 

4 items 

30  items  analyzed 

Item  Difficulty  - number 
of  very  difficult  items 

Lowest  diff  = 53% 
Test  mean  = 67% 

22  items  with  diff 
<50%. 

Test  mean  = 43%. 
Hard  test! 

Lowest  diff=44% 
Test  mean  = 67% 

7 items  with  diff 
<50% 

Test  mean  = 60% 
Not  as  hard  as 
midterm. 

Item  Discrimination  - 
number  of  poorly 
discriminating  items 

All  disc  good  (>.50) 
Expected,  with  only 
3 items. 

8 items  disc  0.0-.15 
3 items  negative 
disc 

1 item  disc  =.17, 
others  >.60 

2 items  disc  0.0-.15 
3 items  negative 
disc. 
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Table  B.9.a.5:  2005  vs.  2006  Test  and  Test  Analysis  Comparisons  - Reinaldo  Moraga 


Midterm 

Final 

Test  Comparisons 

Fall  2005 

Fall  2006 

Fall  2005 

Fall  2006 

Based  on  Item  Bank 

No 

Yes 

No 

Yes 

Based  on  Table  of 
Specifications 

No 

Yes 

No 

Yes 

Overall  Appearance  - well 
laid  out;  pleasing 
appearance,  grammar, 
etc. 

OK 

Take  home? 
Only  2 problems, 
100  pts. 

Great  looking 
exam. 

25  Items,  100  pts. 

OK 

6 problems. 

5 from  textbook, 
100  pts.  Take 
home? 

Part  1 : 3 long 
problems  from  text 
= 100  pts. 

Part  2:  Great 
looking  exam.  25 
Items,  25  pts. 

Overall  Instructions  - 
clear,  unambiguous 

OK 

Good.  More 
comprehensive 

OK 

Good  for  part  2 

Instructions  for  Item 
Subsets  - clear, 
unambiguous 

NA 

NA 

NA 

NA 

Number  of  short,  discrete 
Items  vs  longer  Items 

Both  are  long 
problems. 

All  Items  are  short, 
discrete. 

All  are  long 
problems. 

25  Items  In  part  2 
are  short,  discrete. 

Number  of  objectively 
scored  vs  subjectively 
scored  Items 

Subjectively 

scored. 

All  are  MC  Items. 

Subjectively 

scored. 

3 In  part  1 are 
subjectively 
scored. 

25  In  part  2 are  all 
MC. 

Item  Quality  - clear, 
direct,  well  written,  no 
clues 

OK 

Appears  clear  and 
direct. 

OK 

Good. 

Item  Analysis 
Comparisons 

2 Items  analyzed 

25  Items  analyzed 

6 Items  analyzed 

28  Items  analyzed 

Item  Difficulty  - number 
of  very  difficult  Items 

Both  diff  around 
48% 

7 Items  dIff  < 50% 
Test  mean  = 55% 
(Hard  test) 

All  diff  > 65% 

9 Items  diff  < 50% 
Test  mean  = 60% 

Item  Discrimination  - 
number  of  poorly 
discriminating  items 

Both  disc  around 
.84 

1 Item  disc  0.0-.15 
(a  very  easy  Item) 
1 Item  negative 
disc. 

All  disc  > .46 

4 Items  disc  0.0- 
.15 

6 Items  disc 
negative 
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Table  B.9.a.6:  2005  vs.  2006  Test  and  Test  Analysis  Comparisons  - Regina  Rahn 


Midterm 

Final 

Test  Comparisons 

Fall  2005 

Fall  2006 

Fall  2005 

Fall  2006 

Based  on  Item  Bank 

No 

Yes 

No 

Yes 

Based  on  Table  of 
Specifications 

No 

Yes 

No 

Yes 

Overall  Appearance  - well 
laid  out;  pleasing 
appearance,  grammar, 
etc. 

Good 

5 items  with 
muitipie  parts. 
Totai  of  13 
items/parts. 

Great 

30  items,  100  pts. 

Good 

6 items  with 
multiple  parts. 
(Take  home?) 
Total  of  19 
items/parts 

Good. 

17  items,  105  pts. 

Overall  Instructions  - 
clear,  unambiguous 

No  overali  test 
instructions 

No  overali  test 
instructions 

No  overall  test 
instructions 

No  overall  test 
instructions 

Instructions  for  Item 
Subsets  - clear, 
unambiguous 

Good 

Good 

Good 

Good. 

17  items,  105  pts. 

Number  of  short,  discrete 
items  vs  longer  items 

Most  are  probiem- 
type. 

Some  may  be 
short-answer. 

5 short  answer 
25  MC,  TF, 
Matching 

Most  are  problem- 
type. 

Some  may  be 
short-answer. 

Most  are  short 
answer,  remainder 
are  longer  answer. 

Number  of  objectively 
scored  vs  subjectively 
scored  items 

Ali  are  subjectiveiy 
scored. 

5 subjectively 
scored 

25  objectively 
scored 

All  are  subjectively 
scored. 

All  are  subjectively 
scored. 

Item  Quality  - clear, 
direct,  well  written,  no 
clues 

Good 

Good 

Good 

Good 

Item  Analysis 
Comparisons 

13  items/parts 
anaiyzed 

30  items  analyzed 

6 items  analyzed- 
not  by  parts. 

17  items  analyzed 

Item  Difficulty  - number 
of  very  difficult  items 

2 items  diff  <50% 
Test  mean  = 70% 

3 items  diff  < 50% 
Test  mean  = 78% 

All  diff  >73% 
Test  mean  = 82% 

All  items  diff  > 50% 
Test  mean  = 78% 

Item  Discrimination  - 
number  of  poorly 
discriminating  items 

1 disc  < .15  (=.07) 

5 items  disc  0.0-.15 
2 items  disc 
negative 

1 disc  <.15  (=.06) 
(1  high  scorer  did 
poorly  on  this 
item.) 

1 item  disc  0.0-.15 
3 items  disc 
negative 
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Table  B.9.a.7:  2005  vs.  2006  Test  and  Test  Analysis  Comparisons  - Robert  Tatara 


Midterm 

Final 

Test  Comparisons 

Fall  2005 

Fall  2006 

Fall  2005 

Fall  2006 

Based  on  Item  Bank 

No 

Yes 

No 

Yes 

Based  on  Table  of 
Specifications 

No 

Yes 

No 

Yes 

Overall  Appearance  - well 
laid  out;  pleasing 
appearance,  grammar, 
etc. 

Courier  font. 

25  items  = 100  pts. 

Bigger,  more 
attractive  font. 
Better  overall 
appearance.  30 
items  = 30  pts. 

Courier  font. 
Layout  and 
appearance  OK. 
35  items  = 100  pts. 

Better  overall 
appearance. 
Better  font 

Overall  Instructions  - 
clear,  unambiguous 

Generally  OK 

Much  better, 
clearer,  more 
complete. 

OK 

Much  better, 
clearer,  more 
complete. 

Instructions  for  Item 
Subsets  - clear, 
unambiguous 

NA 

NA 

NA 

NA 

Number  of  short,  discrete 
items  vs  longer  items 

Some  items  are 
longer,  problem 
type.  Some  are 
short  answer. 

No  long  problem- 
type.  Fewer  short 
answer.  More  are 
MC,  TF,  matching. 

Several  short- 
answer.  Some 
longer  problem- 
type.  Not  quite  half 
are  MC,  TF,  etc. 

No  long  problem- 
type.  Fewer  short 
answer.  More  are 
MC,  TF,  matching. 

Number  of  objectively 
scored  vs  subjectively 
scored  items 

Most  are 
subjectively 
scored.  Small 
number  are 
objectively  scored 

Most  are 

objectively  scored  - 
MC,  TF,  matching. 
Very  few  are  short 
answer, 
subjectively 
scored. 

19  subjectively 
scored. 

16  objectively 
scored  - MC,  TF, 
etc. 

Most  are 

objectively  scored  - 
MC,  TF,  matching. 
Very  few  are  short 
answer, 
subjectively 
scored. 

Item  Quality  - clear, 
direct,  well  written,  no 
clues 

Perhaps  some 
slight  ambiguities  -- 
but  mostly  clear 
and  direct. 

Appears  clear  and 
direct 

Generally  good. 

Appears  clear  and 
direct 

Item  Analysis 
Comparisons 

25  items  analyzed 

30  items  analyzed 

35  items  analyzed 

50  items  analyzed 

Item  Difficulty  - number 
of  very  difficult  items 

7 items  (39%)  diff  < 
50% 

Test  mean  = 66% 

9 items  (43%)  diff  < 
50% 

Test  mean  62% 

4 items  (13%)  diff  < 
50% 

Test  mean  74% 

10  items  (20%)  diff 
< 50% 

Test  mean  72% 

Item  Discrimination  - 
number  of  poorly 
discriminating  items 

1 item  disc 
negative 

5 items  disc  0.0-.15 
4 items  disc 
negative 

10  items  disc  0.0- 
.15 

2 items  disc 
negative 

8 items  disc  0.0-.15 
4 items  disc 
negative 
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PERFORMANCE  TASK  AND  RUBRIC  DEVELOPMENT  ASSESSMENT 

Jule  Dee  Scarborough,  Ph.D. 

The  learning  and  professional  growth  on  the  Performance  Assessment  program  or  knowledge 
component  was  measured  by  the  professors’  performance  on  the  task  of  designing  and 
developing  three  complex  performance  tasks  and  three  corresponding  rubrics  for  scoring  task 
achievement.  Using  the  Rubrics  below  as  guiding  criteria,  they  each  designed  three  complex 
performance  tasks  and  corresponding  rubrics.  These  assessments  were  added  to  their  course  a 
new  assessment  strategy  and  assessment  procedures. 

It  is  important  to  note  that  one  performance  task/rubric  was  designed  to  correspond  with  the 
midterm,  and  another  to  correspond  with  the  final  exam  using  the  logic  that  objective  tests 
usually  reflect  what  students  know  or  know  about  rather  than  what  they  can  do.  Therefore, 
we  used  an  unusual  scenario  where  the  professors  “linked”  the  objective  midterm  exam  to  a 
midterm  performance  task/rubric  as  well  as  an  objective  final  exam  to  a final  performance 
task/rubric.  They  also  developed  a third  performance  task/rubric  and  choose  how  and  when 
to  use  it.  They  were  asked  to  “match”  where  they  thought  the  test  items  and  performances 
“overlapped”  and  measured  the  same  or  similar  content.  It  was  assumed  from  studying  the 
literature  that  performance  assessment  measures  different  aspects  of  learning,  sometimes 
deeper  levels  of  learning  through  use  of  knowledge  in  more  active  or  engaging  ways, 
problems,  projects,  etc.  But  performance  assessment  can  also  measure  some  of  the  same 
aspects  of  learning  as  objective  tests.  Also  some  of  the  professors  designed  their  tests  to 
incorporate  some  level  of  performance  in  subjective  or  problem  based  items.  In  examining 
the  tests  and  analyzing  them,  the  objective  items  were  separated  from  the  more  performance 
based  items. 

Professors  were  provided  a presentation  about  Performance  Assessment.  Performance  Tasks 
and  Rubrics  were  discussed,  and  they  received  a portfolio  of  sample  tasks  and  rubrics.  They 
were  given  books  on  the  topic  as  part  of  their  new  library  on  teaching  and  learning.  Their 
performance  tasks  and  rubrics  reflect  the  ABET  or  NAIT  standards  with  corresponding 
rubrics.  Perhaps  one  professor  had  used  simple  and  less  formalized  rubrics  before,  but  none 
of  the  professors  had  developed  or  used  formal,  written,  scenario-based  performance  tasks 
with  corresponding  rubrics  before  this  initiative.  Thus,  there  were  no  previous  instruments  to 
view  from  the  baseline  semester.  Fall  2005,  and  compare  to  these.  Therefore,  we  judged  them 
based  upon  the  Rubrics  below. 

Performance  Task:  Design  and  develop  three  complex  performance  tasks  with 
corresponding  rubrics.  The  tasks  must  be  based  upon  the  ABET  outcomes  or  NAIT 
standards  and  corresponding  student  learning  outcomes  for  the  coure;  they  must  also  reflect 
real  world,  authentic  performances,  tasks,  or  behaviors  in  the  appropriate  community  of 
practice,  e.g.  industry.  The  performance  tasks  and  rubrics  must  be  used  to  measure  student 
learning  in  the  experimental  research  course.  Fall  2006.  See  the  Rubrics  below  for  the 
achievement  criteria  to  use  in  accomplishing  the  task. 
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Performance:  The  professors  accomplished  the  performance  task  well.  The  process  involved 
drafting  initial  and  authentic  real  world  scenarios  with  embedded  task  clusters  and  a 
corresponding  rubric  instrument  for  each  task  (3).  The  program  leader  provided  feedback 
one-on-one  as  the  performance  tasks  were  developed.  The  professors  shared  their  drafts  with 
each  other  and  benefited  from  the  group  critique  process.  The  group  process  worked 
especially  well.  The  tasks  and  rubrics  were  finalized;  the  program  leader  approved  them;  and 
then,  each  professor  used  the  tasks  and  corresponding  rubrics  successfully  with  students 
during  the  2006  experimental  research  semester.  After  the  semester  was  completed,  the 
professors  copied  all  rubrics  returned  to  each  student  in  their  classes  for  all  three  performance 
tasks.  The  program  leader  reviewed  the  scored/with  comments  rubrics  that  each  student 
received  back  from  the  professors.  Thus,  the  use  of  the  rubrics  was  also  reviewed.  Finally, 
the  professors  completed  a feedback/evaluation  form  about  the  use  of  performance 
assessment  for  the  first  time.  As  with  test  analysis  and  development,  the  feedback  from  the 
professors  on  the  value  of  learning  to  design,  develop,  and  use  performance  tasks/rubrics  was 
extremely  positive. 

The  following  rubrics  were  used  to  guide  the  professors  in  the  development  of  the  three 
performance  tasks  and  corresponding  rubrics  for  each  task. 

Also,  the  feedback  and  evaluation  questionnaire  and  professor  responses  are  provided  below, 
following  the  rubrics.  The  faculty  members  truly  felt  that  expanding  their  assessments  to 
include  performance  tasks  with  rubrics  was  extremely  positive.  They  all  indicated  that  they 
will  continue  to  use  performance  assessment,  tasks  and  rubrics,  and  also  expand  the  use  of 
performance  assessment  to  other  courses. 

Responses  to  the  feedback  and  evaluation  related  to  Performance  Assessment  follows  the 
rubrics  below. 
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Rubric  for  Assessing  the  Quality  of  a Performance  Task 

Key  Components  - Properly  Designed  Performance  Tasks  must 

I.  Be  based  on  content  standards  established  by  ABET  or  NAIT 

II.  Describe  a “real-life”  scenario;  are  real  world,  authentic  tasks;  require  active  performances 

III.  Involve  students  in  complex  reasoning  - critical  thinking  at  upper  levels  of  Bloom’s  Cognitive  Dimension 

IV.  Require  students  to  collect  and  process  information,  using  it  for  an  authentic  purpose 

V.  Incorporate  “habits  of  mind” 

VI.  Require  student  collaboration  and  cooperation;  incorporates  “individual”  and  “group”  learning  and  performance 
accountability 

VII.  Result  in  a tangible  product  and/or  communication  activity 

For  each  component,  there  are  descriptors  reflecting  levels  of  achievement  possible: 

I.  The  Performance  Task  is  based  on  the  ABET  or  NAIT  standards 

a.  The  Performance  Task  is  directly  related  to  the  ABET  or  NAIT  standards. 

b.  Learning  standards  are  apparent,  but  the  relation  to  the  task  and/or  national  standards  is  sketchy  or  not  apparent. 

c.  The  Performance  Task  does  not  appear  to  be  based  on  the  standards/outcomes,  course  or  national. 

II.  The  Performance  Task  describes  a “real-life”  scenario  that  is  authentic  and  requires  active  performance. 

a.  The  scenario  described  in  the  task  accurately  mirrors  an  activity  in  the  community  of  practice  outside  the  classroom. 

b.  The  scenario  described  in  the  task  simulates  an  activity  in  the  community  of  practice  outside  the  classroom. 

c.  The  scenario  described  in  the  task  contains  some  aspects  of  activity  outside  the  classroom  but  is  largely  contrived. 

d.  The  scenario  described  in  the  task  is  an  academic  exercise  that  usually  takes  place  only  in  the  context  of  an  academic  setting. 

III.  The  Performance  Task  involves  students  in  complex  reasoning-critical  thinking  processes  at  upper  levels  of  Bloom’s 
Cognitive  Dimension. 

a.  The  task  requires  students  to  utilize  complex  reasoning  - critical  thinking  skills,  such  as  induction/deduction,  diagnosis, 
abstracting,  experimental  inquiry,  problem  solving;  evaluation,  creation,  synthesis,  etc. 

b.  The  task  requires  students  to  utilize  complex  reasoning  components,  such  as  comparing,  classifying,  decision  making,  or 
investigation. 

c.  The  task  requires  students  only  to  recall  facts. 

IV.  The  Performance  Task  requires  students  to  collect  and  process  information,  using  it  for  an  authentic  purpose. 

a.  The  task  incorporates  a variety  of  information  gathering  techniques  and  information  resources.  Students  are  required  to 
interpret  and  synthesize  information  and  accurately  assess  the  value  of  information  gathered.  They  are  required  to  collect  the 
right  information  for  an  authentic  purpose,  e.g.  solve  a problem,  apply  or  use  in  a complex  project,  etc. 

b.  The  task  requires  students  to  gather  and  synthesize  information,  but  the  value  of  the  information  gathered  is  not  assessed. 
Information  may  not  be  used  for  a purpose. 

c.  The  task  requires  the  students  to  gather  information,  but  not  to  interpret  it. 

d.  The  task  requires  no  gathering  or  processing  of  information. 

V.  The  Performance  Task  incorporates  “Habits  of  Mind.” 

a.  The  task  requires  students  to  make  effective  plans,  use  necessary  resources,  evaluate  effectiveness  of  their  own  actions,  seek 
accuracy,  and  engage  in  activities  when  answers  or  solutions  are  not  immediately  apparent. 

b.  The  task  only  requires  students  to  effectively  plan  or  use  resources. 

c.  The  task  does  not  require  students  to  engage  in  self-regulation,  critical,  or  creative  thinking. 

VI.  The  Performance  Task  requires  student  collaboration  and  cooperation;  incorporates  “individual”  and  “group”  learning 
and  performance  accountability. 

a.  The  task  requires  students  to  use  interpersonal  skills,  work  toward  the  achievement  of  team  goals,  and  perform  a variety  of 
roles  within  the  team.  There  is  a formal  team  structure  and  process. 

b.  The  task  requires  students  to  work  together  in  teams  but  there  are  no  measures  described  that  ensure  collaboration  or 
cooperation  among  team  members. 

c.  The  task  is  completed  largely  by  students  on  an  individual  basis  rather  than  in  student  teams. 

VII.  The  Performance  Task  results  in  a tangible  product  and/or  communication  activity. 

a.  The  task  result  is  a tangible  product  or  communication  activity  comparable  to  that  commonly  produced  in  business  or  industry 
community  of  practice. 

b.  The  task  results  in  a product  that  is  similar  to  those  completed  in  business  or  industry  community  of  practice,  but  lacks  several 
components  that  make  the  product  realistic. 

c.  The  task  does  not  result  in  a product  or  communication  activity  relevant  to  a business  or  industry  community  of  practice. 
(Scarborough,  2006  [Based  upon  White  & Scarborough,  2004]) 
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Rubric  for  Assessing  the  Quality  of  a Rubric 


Properly  Designed  Rubrics  Must 

I.  Contain  a set  of  key  components/standards  to  be  assessed  that  reflect  the  student  learning  outcomes 
for  the  course,  which  are  directly  linked  to  the  national  outcomes. 

II.  Include  descriptors  for  each  component/standard  that  are  measurable. 

III.  Have  descriptors-criteria  that  are  indicative  of  observable  student  performances  or  behaviors. 

IV.  Incorporate  a clear  and  well-defined  scoring  system 

V.  (Optional)  Include  appropriate  weights  for  each  component  and  descriptor 

For  each  component,  there  are  descriptors  reflecting  levels  of  achievement  possible: 

I.  The  rubric  contains  a set  of  key  components  (standards)  to  be  assessed. 

a.  A complete  list  of  key  components-standards  is  provided  for  the  performance  task, 
including  the  embedded  subtasks,  if  a cluster.  The  task(s)  are  directly  connected  to 
student  learning  outcomes  for  course  and  the  national  outcomes. 

b.  Key  components/standards  listed  are  not  exhaustive  for  the  performance  task  and/or 

subtasks  embedded  are  not  clear  enough  for  student  response  or  action;  components  or  standards  are  not  clearly 
connected  to  student  learning  outcomes  for  course. 

c.  Not  all  key  components/standards  describe  student  outcomes;  some  are  not  directly  linked  to  national 
outcomes. 

d.  No  key  components  are  listed. 

II.  The  rubric  includes  a set  of  descriptors-criteria  for  each  key  component  or  standard. 

a.  Descriptors-criteria  for  each  component  or  standard  are  arranged  in  a clear  hierarchy  from  non-achievement  to  full- 
achievement. 

b.  Descriptors-criteria  are  present  for  each  component/standard,  but  obvious  levels  in  some  are  missing. 

c.  Each  component  does  not  have  an  associated  set  of  descriptors-criteria. 

III.  The  rubric  descriptors/criteria  are  clear  and  contain  observable  or  measurable  student 
performances  or  behaviors. 

a.  All  descriptors-criteria  clearly  delineate  levels  of  observable  student  performances  or  behaviors. 

b.  Most  descriptors-criteria  clearly  delineate  levels  of  observable  student  performances  or  behaviors. 

c.  Only  a few  descriptors-criteria  clearly  define  levels  of  observable  student  performances  or  behaviors. 

d.  Descriptors-criteria  do  not  describe  observable  student  performances  or  behaviors. 

IV.  Incorporate  a clear  and  well-defined  scoring  system 

a.  There  is  a well  defined  and  clear  system  for  scoring  each  component-standard  and  its  descriptors-  criteria.  Points  or 
percentages  are  assigned  appropriate  to  instructional  and  performance  values. 

b.  The  scoring  system  lacks  definition,  clarity,  and  although  there  is  a scoring  system,  some  aspects  are 
ambiguous,  subjective  or  unclear. 

c.  There  is  no  scoring  system. 

V.  Optional:  Appropriate  weights  are  assigned  to  components  and  descriptors. 

a.  Component-standards  and  descriptors-criteria  are  each  properly  weighted  according  to 

instructional  emphasis  and  performance  values. 

b.  Weights  are  assigned,  but  point  values  do  not  reflect  proper  instructional  emphasis  and  performance  values  in  all 
cases. 

c.  Weights  are  assigned  to  some  performance  standards  and  descriptors,  but  not  others. 


(Scarborough,  2006  [Based  upon  White  & Scarborough,  2004]) 


25 


CEET  Initiative  on  Teaching  and  Learning  - Performance  Assessment  Feedback,  Jan,  2007 

(7/7  respondents) 

1.  Was  the  time  spent  developing  performance  tasks  worth  your  time  - worthwhile? 

(7)  Yes  Not  really 

Why? 

• “A  lot  of  work,  but  they  really  engage  students” 

• “Allowed  me  to  think  about  what  students  should  be  able  to  perform  after  completing  the  course.  ” 

• “Invaluable.  ” “Although  / always  give  ‘projects’,  I was  naive  to  many  of  the  aspects  of  a true 
performance  task.  ” 

• “Performance  tasks  made  students  (1)  solve  open  ended  problems;  (2)  work  in  groups;  (3)  identify 
problems  and  try  to  have  multiple  solutions  and  then  justify  the  solution.  ” 

2.  Was  the  time  spent  developing  rubrics  for  scoring  the  performance  tasks  worth  your  time  - worthwhile? 

(7)  Yes  Not  really 

Why? 

• “Students  knew  what  was  expected  of  them;  grading  was  a bit  easier;  fewer  students  challenged  their 
grades.  ” 

• “Allowed  me  to  set  expectations  from  the  PAs.  ” 

• “The  students  really  responded  well  to  them!  They  liked  knowing  the  expectations  for  performance 
tasks.  ” 

• “I  didn  ’t  have  this  experience  before.  ” 

• “Rubrics  helped  students  understand  what  is  expected  of  them  and  how  they  will  be  graded.  ” 

3.  Would  you  recommend  the  performance  task  program  content  for  other  faculty  members? 

(7)  Yes  Not  really 

Why? 

• “Valuable” 

“It  will  be  a good  exercise  for  others.  ” 

• “1  think  any  new  faculty  should  be  exposed  to  this  experience.  ” 

• “1  feel  students  learned  a lot  because  of  the  performance  tasks.  ” 

4.  Would  you  recommend  the  rubric  program  content  for  other  faculty  members? 

(7)  Yes  Not  really 

Why? 

• “It  will  be  a good  exercise  for  others.  ” 

• “This,  I believe,  is  a necessity.  ” 
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5.  Were  the  performance  tasks  a beneficial  addition  to  the  student  assessment  plan  for  your  course? 

(7)  Yes  Not  really 

Why? 

• “It  was  another  dimension  of  assessment.  Some  students  who  did  not  do  well  on  tests... really  shined  in 
performance.  ” 

“Provides  additional  form  of  assessment  method.  ” 

• “It  added  a new  dimension  of  student  assessment;  also  these  performances  tasks  involved  various 
learning  styles.  ” 

6.  Were  the  rubrics  beneficial  for  scoring  the  performance  tasks? 

(7)  Yes  Not  really 

Why? 

• “Students  knew  what  was  expected  of  them;  grading  was  a bit  easier;  fewer  students  challenged  their 
grades.  ” 

• “Makes  the  scoring  process  easier.  ” 

• “It  made  it  easier  for  me  — and  the  students  also  responded  well.  IT  is  necessary  to  have  a procedure 
mapped  out  for  them  to  understand  the  expectations  and  levels.  ” 

• “They  make  grading  progress  easier.  ” 

• “Otherwise,  it  would  be  very  subjective  or  arbitrary.  ” 

1.  Were  the  performance  tasks  an  effective  tool  for  enhancing  student  learning? 

(7)  Yes  Not  really 

Why? 

• “Bigger,  more  authentic  tasks.  ” 

• “Students  can  demonstrate  what  they  can  perform  after  completing  the  course.  ” 

• “( 1 )It  allowed  for  many  more  teaching  styles  to  be  incorporated  in  the  course;  (2)  more  learning  styles 
were  also  included;  (3)  a good  tool  for  group  work  as  well.  ” 

• “They  really  understand  expectations.  ” 

• “Students’  learning  involves  various  learning  and  teaching  style,  and  models  and  performance  tasks 
provided  these  opportunities.  ” 

8.  Were  the  performance  tasks  an  effective  tool  for  measuring  student  learning? 

(7)  Yes  Not  really 

Why? 

• “It  was  another  dimension  of  assessment.  Some  students  who  did  not  do  well  on  tests... really  shined  in 
performance.  ” 

• “Students  can  demonstrate  what  they  can  perform  after  completing  the  course.  ” 

• “(1)  It  demonstrated  their  abilities  to  communicate  effectively;  (2)  It  demonstrated  their  abilities  to 
synthesize,  apply,  and  evaluate  subject  matter  content.  ” 

• “Students’  learning  may  not  be  completely  assessed  by  only  exams  and  homework.  ” 
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9.  Were  the  rubrics  an  effective  tool  for  scoring  the  outcomes  of  student  performances  on  the  tasks? 

(7)  Yes  Not  really 

Why? 

• “It  was  another  dimension  of  assessment.  Some  students  who  did  not  do  well  on  tests... really  shined  in 
performance.  ” 

• “See  previous  comments.  ” 

• “Rubrics  provide  the  details  of  expected  outcomes.  ” 

10.  Were  the  rubrics  effective  in  helping  students  to  understand  more  about  what  you  expected  them  to  do 
by  revealing  the  standards  and  scoring  mechanism  with  them  up  front? 

(7)  Yes  Not  really 

Why? 

• “It  was  another  dimension  of  assessment.  Some  students  who  did  not  do  well  on  tests... really  shined  in 
performance.  ” 

• “Students  know  the  expectations.  ” 

• “See  previous  comments.  ” 

• “They  know  what  is  expected  of  them.  ” 

11.  Do  you  feel  that  more  formalized  performance  tasks  and  rubrics  improve  the  opportunity  for  students  to 
provide  evidence  of  learning? 

(6)  Yes  (1)  Not  really 

Why? 

• “It  was  another  dimension  of  assessment.  Some  students  who  did  not  do  well  on  tests... really  shined  in 
performance.  ” 

• “Not  everyone  is  good  in  taking  tests.  Also  exams  and  homework  do  not  provide  the  opportunity 
through  performance  tasks.  ” 

• “Two  is  enough.  ” 

12.  Would  you  recommend  that  other  faculty  members  get  the  opportunity  to  learn  to  develop  and  use 
performance  tasks  and  rubrics  as  student  assessment  tools? 

(7)  Yes  Not  really 

Why? 

• “I  believe  this  was  one  of  the  most  beneficial  aspects  of  the  program  with  regard  to  student  learning 
and  assessment.  It  ties  in  with  active  learning  and  Bloom ’s  Taxonomy.  ” 

• “It  was  a big  help  for  me.” 
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13.  Was  the  performance/rubric  development  process  used  with  this  group  - “developing  while  learning  from 
presentation,  examples,  and  one-on-one  feedback”  - effective? 

(7)  Yes  Not  really 

Why? 

• “ One-on-one  feedback  especially  helpful.  ” 

14.  Will  you  continue  to  use  performance  tasks  and  rubrics  in  this  and/or  other  classes? 

(7)  Yes  Not  really 

Why? 

• “To  improve  student  learning.  ” 

15.  Strengths  of  the  performance  task/rubric  program  component. 

• “Already  stated  in  above  [ responses].  ” 

• “Measures  what  students  can  really  perform  with  their  learned  tools.  ” 

• “Quantified  student  performance;  gave  students  guidance  and  goals.  ” 

• “I  liked  the  development  of  the  Performance  Tasks,  especially  with  the  rubric.  Discussions  were 
enlightening,  as  well  as  our  group  discussions  and  evaluations.  ” 

• “Very  good  way  in  presenting  material;  Different  styles  of  rubrics  presented;  also  working  in  our  same 
classes  helped  to  learn  how  to  do  rubrics  and  performances.  ” 

• “Provide  other  teaching  styles,  learning  styles,  and  teaching  models.  ” 

• “Allow  for  active  learning.  Results  show  improvement  when  Performance  Tasks  were  done  in 
groups.  ” 

16.  Areas  to  improve  in  the  performance  task/rubric  program  component. 

• “Revisit  and  revise.  ” 

• “None.  ” 

• “Streamline  the  time  scale.  ” 

• “Good  as  is.  ” 

17.  General  comments: 

• “This  part  was  exceptional — I will  always  use  this  info  in  my  classes  in  the  future.  ” 

• “Very  good  program.  ” 

• “Results  indicate  conclusively  that  learning  level  was  enhanced  [by  students  in  experimental 
semester].  ” 
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MIDTERM  AND  FINAL  EXAM  ^ PERFORMANCE  ASSESSMENT 

CORRELATION^ 

JULE  DEE  SCARBOROUGH,  PH.D.  AND  JERRY  GILMER,  PH.D. 

Typically,  traditional  objective  tests  are  only  indicators  of  what  students  can  do  with  the 
knowledge  being  measured.  Performing  well  on  a traditional  test  should  not  lead  to  an 
assumption  of  what  a student  can  do  with  that  knowledge  (e.g.,  how  well  they  can  use  the 
knowledge).  Traditional  or  objective  tests  usually  measure  what  students  know  or  know 
about,  while  performance  assessments  engage  students  in  performance  tasks  where  they 
use  the  knowledge  in  some  way.  It  is  sometimes  perceived  by  performance  assessment 
advocates  that  performance  assessments  (if  designed,  developed,  and  well  constructed) 
are  better  evidence  of  what  students  are  capable  of  doing  with  knowledge  gained.  That  is 
assuming  that  most  traditional  tests  are  written  to  measure  memory  for  information, 
concepts,  theories,  facts.  If,  however,  tests  have  been  written  to  include  items  that  are 
higher  on  Bloom’s  Taxonomy  and  require  more  critical  thinking  or  problem  solving,  then 
those  tests  would  provide  evidence  of  something  more  than  what  students  know  about,  as 
the  particular  items  engaging  students  in  higher  levels  of  critical  thinking  and  problems 
solving  usually  require  that  students  provide  evidence  of  what  they  know  about  by  using 
that  knowledge  to  solve  the  problem.  That  is,  if  the  problems  are  complex  and  well 
constructed,  use  of  the  knowledge  will  provide  evidence  of  learning. 

Some  prefer  to  use  tests  intentionally  as  indicators  of  what  students  know  about  and  then 
follow  those  tests  with  performance  tasks  requiring  students  to  solve  problems  or  engage 
in  projects  that  require  critical  thinking,  the  manipulation  of  facts,  theories,  concepts, 
and/or  information  in  a context  where  particular  constraints  and  conditions  as  well  as 
tools,  procedures,  etc.  are  set.  If  this  is  the  goal,  then  a test  and  performance  task(s)  may 
be  designed  to  include  measurement  of  some  of  the  same  content  while  also  measuring 
some  different  content,  as  they  are  distinctly  different  types  of  measures  with  the 
potential  to  accomplish  different  measurement  goals  as  well  as  some  of  the  same  goals. 
Therefore,  we  asked  the  professors  to  design  and  develop  an  objective  midterm  and  final 
examination  as  well  as  corresponding  performance  task(s)  “and  scoring  rubrics 
matching”  the  content  where  possible  or  desirable.  The  professors  were  also  asked  to 
identify  the  objective  test  items  they  felt  were  also  being  measured  on  the  corresponding 
performance  task(s). 

A statistical  correlation  was  run  between  the  midterm  exam  and  corresponding 
performance  assessment  and  the  final  exam  and  corresponding  performance  examination 
for  each  professor’s  students.  The  results  should  lead  the  professors  to  consider  the 
following: 

1.  Do  they  really  feel  that  there  is  a segment  of  the  objective  tests  and  the 
performance  tasks  where  there  is  a content  match?  If  so,  in  our  program,  no 
external  contentvalidation  was  required.  We  assumed  the  professors  knew 


Note:  Correlations  have  been  computed  in  two  ways:  1 ) leaving  zero  scores  in  as  zeros;  2 ) replacing 
zero  scores  with  blanks  or  taking  them  out,  e.g.  student  was  absent. 
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their  content.  However,  it  is  key  to  note  that  it  is  important  for  professors 
using  any  measurement  procedure  or  tool  to  validate  content,  procedures,  etc. 
externally  in  the  purest  sense  of  measurement  or  student  assessment.  That, 
however,  takes  more  time  to  execute  with  a faculty  learning  community  and, 
in  our  opinion,  would  be  part  of  a Stage  II  faculty  development  program.  Our 
focus  was  on  test  analysis,  writing  better  objective  tests  as  well  as  better  and 
higher  level  items  to  include  problem  solving  items.  In  addition,  our  program 
focused  on  introducing  them  to  the  design,  development,  and  use  of 
performance  tasks  and  rubrics  as  one  type  of  learning  measurement  procedure 
or  tool. 

2.  How  are  professors  using  the  tests  and  performance  task(s)?  In  our  case,  we 
encouraged  them  to  design  new  tests  with  more  items,  a wider  range  of  item 
types,  and  items  that  offer  the  opportunity  to  perform  at  various  levels  of 
Bloom’s  learning  (e.g.,  memory  to  synthesis).  We  then  asked  them  to  design 
and  develop  corresponding  performance  tasks  and  rubrics  to  provide  students 
the  opportunity  to  provide  evidence  of  learning  through  performances. 

a.  Do  they  feel  that  the  objective  tests  are  indicators  of  what  students 
know  and  the  performance  tasks  take  the  students  to  the  next  level 
where  they  are  positioned  to  more  deeply  or  critically  use  the 
knowledge  measured  on  the  objective  tests? 

b.  Do  they  feel  that  they  can  better  measure  some  types  of  knowledge 
with  objective  tests  and  other  types  of  knowledge  through 
performances? 

c.  Other  considerations 

3.  What  might  the  correlation  scores  mean?  How  can  they  be  used? 

a.  The  correlation  scores  might  have  no  or  little  meaning. 

b.  The  scores  might  provide  insight  about  students. 

c.  The  scores  might  stimulate  diagnostic  thoughts  about  student 
assessment. 
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Table  B.9.C.1: 

Professor  = Ibrahim  Abdel-Motaleb 


Correlations(b)  - Including  Zero  Scores 


Midterm 

Final 

PA1 

PA2 

PAS 

Pearson  Correlation 

1 

.(a) 

0.263 

.(a) 

.(a) 

Midterm  Sig.  (2-tailed) 

0.238 

N 

24 

0 

22 

0 

0 

Pearson  Correlation 

.(a) 

.(a) 

.(a) 

.(a) 

.(a) 

Final  Sig.  (2-tailed) 

N 

0 

0 

0 

0 

0 

Pearson  Correlation 

0.263 

.(a) 

1 

.(a) 

.(a) 

PA1  Sig.  (2-tailed) 

0.238 

N 

22 

0 

22 

0 

0 

Pearson  Correlation 

.(a) 

.(a) 

.(a) 

.(a) 

.(a) 

PA2  Sig.  (2-tailed) 

N 

0 

0 

0 

0 

0 

Pearson  Correlation 

.(a) 

.(a) 

.(a) 

.(a) 

.(a) 

PAS  Sig.  (2-tailed) 

N 

0 

0 

0 

0 

0 

a.  Cannot  be  computed  because  at  least  one  of  the  variables  is  constant. 

b.  Professor  = Ibrahim  Abdel-Motaleb 


Correlations(b) 

1 - With  Blanks  Replacing  Zeros 

Midterm 

Final 

PA1 

PA2 

PAS 

Pearson  Correlation 

1 

.(a) 

0.263 

.(a) 

.(a) 

Midterm  Sig.  (2-tailed) 

0.238 

N 

24 

0 

22 

0 

0 

Pearson  Correlation 

.(a) 

.(a) 

.(a) 

.(a) 

.(a) 

Final  Sig.  (2-tailed) 

N 

0 

0 

0 

0 

0 

Pearson  Correlation 

0.263 

.(a) 

1 

.(a) 

.(a) 

PA1  Sig.  (2-tailed) 

0.238 

N 

22 

0 

22 

0 

0 

Pearson  Correlation 

.(a) 

.(a) 

.(a) 

.(a) 

.(a) 

PA2  Sig.  (2-tailed) 

N 

0 

0 

0 

0 

0 

Pearson  Correlation 

.(a) 

.(a) 

.(a) 

.(a) 

.(a) 

PA3  Sig.  (2-tailed) 

N 

0 

0 

0 

0 

0 

a.  Cannot  be  computed  because  at  least  one  of  the  variables  is  constant. 

b.  Professor  = Ibrahim  Abdel-Motaleb 
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Table  B.9.C.2: 

Professor  = Abul  Azad 


Correlations(a)  - Including  Zero  Scores 


Midterm 

Final 

PA1 

PA2 

PAS 

Pearson  Correlation 

1 

0.388 

.739(**) 

0.509 

.61 4(*) 

Midterm  SIg.  (2-talled) 

0.153 

0.002 

0.053 

0.015 

N 

15 

15 

15 

15 

15 

Pearson  Correlation 

0.388 

1.000 

.592(*) 

0.430 

.604(*) 

Final  Sig.  (2-tailed) 

0.153 

0.020 

0.110 

0.017 

N 

15 

15 

15 

15 

15 

Pearson  Correlation 

.739(**) 

.592(*) 

1 

.526(*) 

.922(**) 

PA1  Sig.  (2-tailed) 

0.002 

0.020 

0.044 

0.000 

N 

15 

15 

15 

15 

15 

Pearson  Correlation 

0.509 

0.430 

.526(*) 

1.000 

0.410 

PA2  Sig.  (2-tailed) 

0.053 

0.110 

0.044 

0.129 

N 

15 

15 

15 

15 

15 

Pearson  Correlation 

.61 4(*) 

.604(*) 

.922(**) 

0.410 

1.000 

PAS  Sig.  (2-tailed) 

0.015 

0.017 

0.000 

0.129 

N 

15 

15 

15 

15 

15 

**.  Correlation  is  significant  at  the  0.01  level  (2-talled). 
*.  Correlation  Is  significant  at  the  0.05  level  (2-tailed), 
a.  Professor  = Abul  Azad 


Correlations(a) 

- With  Blanks  Replacing  Zeros 

Midterm 

Final 

PA1 

PA2 

PAS 

Pearson  Correlation 

1 

0.425 

.739(**) 

0.509 

-0.114 

Midterm  Sig.  (2-tailed) 

0.130 

0.002 

0.053 

0.698 

N 

15 

14 

15 

15 

14 

Pearson  Correlation 

0.425 

1.000 

.759(**) 

0.327 

0.483 

Final  Sig.  (2-tailed) 

0.130 

0.002 

0.254 

0.094 

N 

14 

14 

14 

14 

13 

Pearson  Correlation 

.739(**) 

.759(**) 

1 

.526(*) 

0.202 

PA1  Sig.  (2-tailed) 

0.002 

0.002 

0.044 

0.489 

N 

15 

14 

15 

15 

14 

Pearson  Correlation 

0.509 

0.327 

.526(*) 

1.000 

0.061 

PA2  Sig.  (2-tailed) 

0.053 

0.254 

0.044 

0.835 

N 

15 

14 

15 

15 

14 

Pearson  Correlation 

-0.114 

0.483 

0.202 

0.061 

1.000 

PAS  Sig.  (2-tailed) 

0.698 

0.094 

0.489 

0.835 

N 

14 

13 

14 

14 

14 

**.  Correlation  is  significant  at  the  0.01  level  (2-talled). 
*.  Correlation  Is  significant  at  the  0.05  level  (2-tailed), 
a.  Professor  = Abul  Azad 
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Table  B.9.C.3: 

Professor  = Brianno  Coller 


Correlations(a)  - Including  Zero  Scores 


Midterm 

Final 

PA1 

PA2 

PAS 

Pearson  Correlation 

1 

.61 0(**) 

.390(**) 

0.159 

.320(**) 

Midterm  SIg.  (2-talled) 

0.000 

0.000 

0.165 

0.004 

N 

78 

78 

78 

78 

78 

Pearson  Correlation 

.61 0(**) 

1.000 

.41 4(**) 

0.134 

.805(**) 

Final  Sig.  (2-tailed) 

0.000 

0.000 

0.242 

0.000 

N 

78 

78 

78 

78 

78 

Pearson  Correlation 

.390(**) 

.41 4(**) 

1 

0.156 

.330(**) 

PA1  Sig.  (2-tailed) 

0.000 

0.000 

0.172 

0.003 

N 

78 

78 

78 

78 

78 

Pearson  Correlation 

0.159 

0.134 

0.156 

1.000 

0.111 

PA2  Sig.  (2-tailed) 

0.165 

0.242 

0.172 

0.333 

N 

78 

78 

78 

78 

78 

Pearson  Correlation 

.320(**) 

.805(**) 

.330(**) 

0.111 

1.000 

PAS  Sig.  (2-tailed) 

0.004 

0.000 

0.003 

0.333 

N 

78 

78 

78 

78 

78 

**.  Correlation  is  significant  at  the  0.01  level  (2-talled). 
a.  Professor  = Brianno  Coller 


Correlations(a] 

1 - With  Blanks  Replacing  Zeros 

Midterm 

Final 

PA1 

PA2 

PAS 

Pearson  Correlation 

1 

.71 1(**) 

.371(**) 

0.159 

.257(*) 

Midterm  Sig.  (2-tailed) 

0.000 

0.001 

0.165 

0.028 

N 

78 

71 

76 

78 

73 

Pearson  Correlation 

.71 1(**) 

1.000 

.331(**) 

0.075 

0.168 

Final  Sig.  (2-tailed) 

0.000 

0.005 

0.536 

0.161 

N 

71 

71 

70 

71 

71 

Pearson  Correlation 

.371(**) 

.331(**) 

1 

0.135 

.31 8(**) 

PA1  Sig.  (2-tailed) 

0.001 

0.005 

0.245 

0.006 

N 

76 

70 

76 

76 

72 

Pearson  Correlation 

0.159 

0.075 

0.135 

1.000 

0.131 

PA2  Sig.  (2-tailed) 

0.165 

0.536 

0.245 

0.270 

N 

78 

71 

76 

78 

73 

Pearson  Correlation 

.257(*) 

0.168 

.31 8(**) 

0.131 

1.000 

PA3  Sig.  (2-tailed) 

0.028 

0.161 

0.006 

0.270 

N 

73 

71 

72 

73 

73 

**.  Correlation  is  significant  at  the  0.01  level  {2-talled). 
*.  Correlation  Is  significant  at  the  0.05  level  (2-tailed), 
a.  Professor  = Brianno  Coller 
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Table  B.9.C.4: 

Professor  = Abhijit  Gupta 


Correlations(a)  - Including  Zero  Scores 


Midterm 

Final 

PA1 

PA2 

PAS 

Pearson  Correlation 

1 

.544(**) 

0.081 

-0.056 

-0.123 

Midterm  Sig.  (2-tailed) 

0.000 

0.602 

0.716 

0.425 

N 

44 

44 

44 

44 

44 

Pearson  Correlation 

.544(**) 

1.000 

.304(*) 

0.045 

0.076 

Final  Sig.  (2-tailed) 

0.000 

0.045 

0.773 

0.626 

N 

44 

44 

44 

44 

44 

Pearson  Correlation 

0.081 

.304(*) 

1 

-0.045 

-0.073 

PA1  Sig.  (2-tailed) 

0.602 

0.045 

0.773 

0.638 

N 

44 

44 

44 

44 

44 

Pearson  Correlation 

-0.056 

0.045 

-0.045 

1.000 

-0.003 

PA2  Sig.  (2-tailed) 

0.716 

0.773 

0.773 

0.982 

N 

44 

44 

44 

44 

44 

Pearson  Correlation 

-0.123 

0.076 

-0.073 

-0.003 

1.000 

PAS  Sig.  (2-tailed) 

0.425 

0.626 

0.638 

0.982 

N 

44 

44 

44 

44 

44 

**.  Correlation  is  significant  at  the  0.01  level  (2-tailed). 
*.  Correlation  is  significant  at  the  0.05  level  (2-tailed), 
a.  Professor  = Abhijit  Gupta 


Correiations(a) 

- With  Blanks  Replacing  Zeros 

Midterm 

Final 

PA1 

PA2 

PAS 

Pearson  Correlation 

1 

.544(**) 

0.081 

-0.056 

-0.123 

Midterm  Sig.  (2-tailed) 

0.000 

0.602 

0.716 

0.425 

N 

44 

44 

44 

44 

44 

Pearson  Correlation 

.544(**) 

1.000 

.304(*) 

0.045 

0.076 

Final  Sig.  (2-tailed) 

0.000 

0.045 

0.773 

0.626 

N 

44 

44 

44 

44 

44 

Pearson  Correlation 

0.081 

.304(*) 

1 

-0.045 

-0.073 

PA1  Sig.  (2-tailed) 

0.602 

0.045 

0.773 

0.638 

N 

44 

44 

44 

44 

44 

Pearson  Correlation 

-0.056 

0.045 

-0.045 

1.000 

-0.003 

PA2  Sig.  (2-tailed) 

0.716 

0.773 

0.773 

0.982 

N 

44 

44 

44 

44 

44 

Pearson  Correlation 

-0.123 

0.076 

-0.073 

-0.003 

1.000 

PAS  Sig.  (2-tailed) 

0.425 

0.626 

0.638 

0.982 

N 

44 

44 

44 

44 

44 

**.  Correlation  is  significant  at  the  0.01  level  (2-tailed). 
*.  Correlation  is  significant  at  the  0.05  level  (2-tailed), 
a.  Professor  = Abhijit  Gupta 
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Table  B.9.C.5: 

Professor  = Reinaldo  Moraga 


Correlations(a)  - Including  Zero  Scores 


Midterm 

Final 

PA1 

PA2 

PAS 

Pearson  Correlation 

1 

.777(**) 

.532(*) 

0.178 

0.130 

Midterm  Sig.  (2-tailed) 

0.000 

0.023 

0.479 

0.608 

N 

18 

18 

18 

18 

18 

Pearson  Correlation 

.777(**) 

1.000 

0.149 

0.293 

0.110 

Final  Sig.  (2-tailed) 

0.000 

0.554 

0.238 

0.663 

N 

18 

18 

18 

18 

18 

Pearson  Correlation 

.532(*) 

0.149 

1 

0.074 

0.214 

PA1  Sig.  (2-tailed) 

0.023 

0.554 

0.771 

0.395 

N 

18 

18 

18 

18 

18 

Pearson  Correlation 

0.178 

0.293 

0.074 

1.000 

.874(**) 

PA2  Sig.  (2-tailed) 

0.479 

0.238 

0.771 

0.000 

N 

18 

18 

18 

18 

18 

Pearson  Correlation 

0.130 

0.110 

0.214 

.874(**) 

1.000 

PAS  Sig.  (2-tailed) 

0.608 

0.663 

0.395 

0.000 

N 

18 

18 

18 

18 

18 

**.  Correlation  is  significant  at  the  0.01  level  (2-tailed). 
*.  Correlation  is  significant  at  the  0.05  level  (2-tailed), 
a.  Professor  = Reinaldo  Moraga 


Correiations(a) 

- With  Blanks  Repiacing  Zeros 

Midterm 

Final 

PA1 

PA2 

PAS 

Pearson  Correlation 

1 

.777(**) 

.532(*) 

0.186 

-0.331 

Midterm  Sig.  (2-tailed) 

0.000 

0.023 

0.490 

0.194 

N 

18 

18 

18 

16 

17 

Pearson  Correlation 

.777(**) 

1.000 

0.149 

0.458 

-0.241 

Final  Sig.  (2-tailed) 

0.000 

0.554 

0.075 

0.352 

N 

18 

18 

18 

16 

17 

Pearson  Correlation 

.532(*) 

0.149 

1 

-0.177 

-0.171 

PA1  Sig.  (2-tailed) 

0.023 

0.554 

0.511 

0.511 

N 

18 

18 

18 

16 

17 

Pearson  Correlation 

0.186 

0.458 

-0.177 

1.000 

0.254 

PA2  Sig.  (2-tailed) 

0.490 

0.075 

0.511 

0.343 

N 

16 

16 

16 

16 

16 

Pearson  Correlation 

-0.331 

-0.241 

-0.171 

0.254 

1.000 

PAS  Sig.  (2-tailed) 

0.194 

0.352 

0.511 

0.343 

N 

17 

17 

17 

16 

17 

**.  Correlation  is  significant  at  the  0.01  level  (2-tailed). 
*.  Correlation  is  significant  at  the  0.05  level  (2-tailed), 
a.  Professor  = Reinaldo  Moraga 
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Table  B.9.C.6: 

Professor  = Regina  Rahn 


Correlations(a)  - Including  Zero  Scores 


Midterm 

Final 

PA1 

PA2 

PAS 

Pearson  Correlation 

1 

.626(*) 

0.510 

.662(**) 

.590(*) 

Midterm  Sig.  (2-tailed) 

0.017 

0.062 

0.010 

0.026 

N 

14 

14 

14 

14 

14 

Pearson  Correlation 

.626(*) 

1.000 

.575(*) 

.637(*) 

.560(*) 

Final  Sig.  (2-tailed) 

0.017 

0.031 

0.014 

0.037 

N 

14 

14 

14 

14 

14 

Pearson  Correlation 

0.510 

.575(*) 

1 

0.478 

0.375 

PA1  Sig.  (2-tailed) 

0.062 

0.031 

0.084 

0.186 

N 

14 

14 

14 

14 

14 

Pearson  Correlation 

.662(**) 

.637(*) 

0.478 

1.000 

.785(**) 

PA2  Sig.  (2-tailed) 

0.010 

0.014 

0.084 

0.001 

N 

14 

14 

14 

14 

14 

Pearson  Correlation 

.590(*) 

.560(*) 

0.375 

.785(**) 

1.000 

PAS  Sig.  (2-tailed) 

0.026 

0.037 

0.186 

0.001 

N 

14 

14 

14 

14 

14 

*.  Correlation  is  significant  at  the  0.05  level  (2-tailed). 
**.  Correlation  is  significant  at  the  0.01  level  (2-tailed), 
a.  Professor  = Regina  Rahn 


Correiations(a] 

1 - With  Blanks  Repiacing  Zeros 

Midterm 

Final 

PA1 

PA2 

PAS 

Pearson  Correlation 

1 

.626(*) 

0.510 

.662(**) 

.590(*) 

Midterm  Sig.  (2-tailed) 

0.017 

0.062 

0.010 

0.026 

N 

14 

14 

14 

14 

14 

Pearson  Correlation 

.626(*) 

1.000 

.575(*) 

.637(*) 

.560(*) 

Final  Sig.  (2-tailed) 

0.017 

0.031 

0.014 

0.037 

N 

14 

14 

14 

14 

14 

Pearson  Correlation 

0.510 

.575(*) 

1 

0.478 

0.375 

PA1  Sig.  (2-tailed) 

0.062 

0.031 

0.084 

0.186 

N 

14 

14 

14 

14 

14 

Pearson  Correlation 

.662(**) 

.637(*) 

0.478 

1.000 

.785(**) 

PA2  Sig.  (2-tailed) 

0.010 

0.014 

0.084 

0.001 

N 

14 

14 

14 

14 

14 

Pearson  Correlation 

.590(*) 

.560(*) 

0.375 

.785(**) 

1.000 

PAS  Sig.  (2-tailed) 

0.026 

0.037 

0.186 

0.001 

N 

14 

14 

14 

14 

14 

*.  Correlation  is  significant  at  the  0.05  level  (2-tailed). 
**.  Correlation  is  significant  at  the  0.01  level  (2-tailed), 
a.  Professor  = Regina  Rahn 
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Table  B.9.C.7: 

Professor  = Robert  Tatars 


Correlations(a)  - Including  Zero  Scores 


Midterm 

Final 

PA1 

PA2 

PAS 

Pearson  Correlation 

1 

.425(*) 

-0.126 

0.101 

0.209 

Midterm  Sig.  (2-tailed) 

0.024 

0.524 

0.608 

0.285 

N 

28 

28 

28 

28 

28 

Pearson  Correlation 

.425(*) 

1.000 

-0.099 

.543(**) 

.620(**) 

Final  Sig.  (2-tailed) 

0.024 

0.618 

0.003 

0.000 

N 

28 

28 

28 

28 

28 

Pearson  Correlation 

-0.126 

-0.099 

1 

0.311 

-0.003 

PA1  Sig.  (2-tailed) 

0.524 

0.618 

0.108 

0.987 

N 

28 

28 

28 

28 

28 

Pearson  Correlation 

0.101 

.543(**) 

0.311 

1.000 

0.310 

PA2  Sig.  (2-tailed) 

0.608 

0.003 

0.108 

0.109 

N 

28 

28 

28 

28 

28 

Pearson  Correlation 

0.209 

.620(**) 

-0.003 

0.310 

1.000 

PAS  Sig.  (2-tailed) 

0.285 

0.000 

0.987 

0.109 

N 

28 

28 

28 

28 

28 

*.  Correlation  is  significant  at  the  0.05  level  (2-tailed). 
**.  Correlation  is  significant  at  the  0.01  level  (2-tailed), 
a.  Professor  = Robert  Tatara 


Correiations(a] 

1 - With  Blanks  Repiacing  Zeros 

Midterm 

Final 

PA1 

PA2 

PAS 

Pearson  Correlation 

1 

.425(*) 

-0.126 

0.101 

0.209 

Midterm  Sig.  (2-tailed) 

0.024 

0.524 

0.608 

0.285 

N 

28 

28 

28 

28 

28 

Pearson  Correlation 

.425(*) 

1.000 

-0.099 

.543(**) 

.620(**) 

Final  Sig.  (2-tailed) 

0.024 

0.618 

0.003 

0.000 

N 

28 

28 

28 

28 

28 

Pearson  Correlation 

-0.126 

-0.099 

1 

0.311 

-0.003 

PA1  Sig.  (2-tailed) 

0.524 

0.618 

0.108 

0.987 

N 

28 

28 

28 

28 

28 

Pearson  Correlation 

0.101 

.543(**) 

0.311 

1.000 

0.310 

PA2  Sig.  (2-tailed) 

0.608 

0.003 

0.108 

0.109 

N 

28 

28 

28 

28 

28 

Pearson  Correlation 

0.209 

.620(**) 

-0.003 

0.310 

1.000 

PAS  Sig.  (2-tailed) 

0.285 

0.000 

0.987 

0.109 

N 

28 

28 

28 

28 

28 

*.  Correlation  is  significant  at  the  0.05  level  (2-tailed). 
**.  Correlation  is  significant  at  the  0.01  level  (2-tailed), 
a.  Professor  = Robert  Tatara 
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DIAGNOSTIC  WRITE  UPS  FROM  TEST  ANALYSES  AND  PERFORMANCE 
TASK/RUBRIC  REVIEWS  BY  PROFESSORS 
Jule  Dee  Scarborough,  Ph.D. 

The  professors  were  asked  to  consider  the  results  of  the  test  analyses  for  the  2005 
midterm  and  final  exams,  use  the  results  as  a diagnostic  procedure  by  critically  reflecting 
on  the  results  of  each  analysis,  and  determine  each  test’s  strength  and  weaknesses. 
Finally,  the  professors  were  asked  to  identify  changes  that  would  improve  the  two  tests. 
This  “closed  the  loop”;  in  other  words,  they  began  to  realize  the  ultimate  purpose  of 
performing  test  analysis  - that  of  identifying  and  making  changes  to  improve  the  quality 
and  process  of  their  testing  and  ultimately  to  improve  instruction  and  student  learning. 
This  also  served  to  provide  a full  circle  of  experience  with  test  analysis  and  its  diagnostic 
and  change  process,  as  they  performed  their  initial  test  analyses  on  the  Fall  2005  tests. 

For  their  first  experience,  if  they  could  not  use  their  own  tests  because  of  no  objective 
items,  then  a real  case  was  provided  in  place  of  their  own  for  the  Fall  2005  analysis. 
Using  the  diagnostic  information  generated  by  the  analyses,  they  developed  new  midterm 
and  final  exams,  following  each  with  a test  analysis  and  diagnostic  write  up. 

After  the  professors  implemented  the  newly  developed  performance  assessments  1,  2, 
and  3,  they  were  asked  to  also  review  and  critically  reflect  upon  their  content  and  use. 
This  provided  another  opportunity  to  engage  in  critical  reflection  (Reflective  Practice) 
and  for  the  first  time  on  performance  assessment.  As  they  had  never  used  performance 
assessments  and  rubrics  as  an  assessment  procedure  before,  this  was  a new  academic 
adventure.  Thus,  they  reviewed  the  results  stemming  from  the  critical  reflection, 
considered  content  and  process,  and  wrote  up  their  thoughts  on  the  performance 
assessment  implementation  during  the  2006  research  semester,  identifying  what  they 
would  change  for  the  next  use  of  the  performance  assessments  and  rubrics. 

Each  did  so.  The  review  formats  and  descriptions  below  range  from  informal  to  more 
formal.  The  style  or  format  used  to  document  their  thoughts  was  not  so  important. 

Instead,  in  addition  to  adding  tools  to  their  teaching  and  learning  practice,  the  process  of 
Reflective  Practice,  critical  analysis,  and  ongoing  analysis  and  change  was  the  ultimate 
purpose  of  the  process.  We  have  quoted  them  below. 
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Abul  Azad 
Diagnostic  Write-up 

Performance  Assessment  1 (PAl) 

The  first  PA  for  this  course  is  based  upon  the  initial  topics  that  have  been  covered 
within  the  first  few  weeks  of  the  class;  this  is  known  as  the  PAl.  Within  this  PA,  students 
needed  to  design  a simple  electronic  system  while  considering  a number  of  design 
factors.  During  the  implementation,  students  needed  to  consider  a number  of  pre- 
conditions. This  was  something  new  for  the  students  of  this  course  because  students  were 
not  familiar  with  the  rubric  system,  and  it  was  difficult  to  introduce  this. 

Midterm 

The  midterm  contains  both  subjective  and  objective  (multiple  choices)  items. 
Considering  the  relatively  smaller  mark  allocation  for  the  objective  type  items,  it  was 
possible  to  cover  more  subject  areas. 

Test  analysis  reveals  the  impact  of  the  items  in  terms  of  answering  by  the 
students.  The  item  difficulty  varied  within  73%  to  100%,  with  most  of  them  above  90%. 
Considering  this,  the  items  could  be  little  more  challenging.  Also  the  item  discrimination 
is  relatively  lower  for  the  objective  items,  with  one  having  a negative  value  and  one 
divided  by  zero. 

After  a review  of  the  midterm  items,  it  appears  that  all  the  items  are  addressing 
important  course  topics  and  these  should  be  a part  of  the  question  bank  (that  has  already 
been  developed).  Also  it  may  be  a good  idea  to  make  the  items  a little  more  challenging 
to  reduce  the  item  difficulty  factor,  which  will  help  to  differentiate  between  students’ 
abilities  in  terms  of  their  performance.  To  improve  the  item  discrimination,  the  items  can 
be  reviewed  for  their  presentation  and  also  to  expose  students  to  items  of  similar  nature 
through  quizzes  and  course  review  process. 

Midterm  Performance  Assessment 

The  PA  that  linked  to  the  midterm  was  PA2.  The  students  taking  the  course  have 
just  started  their  coursework  with  the  Electrical  Engineering  Technology  program,  and 
most  of  them  do  not  have  any  experience  with  the  performance  assessment  and 
assessment  rubrics.  So  it  was  a challenge  for  the  students  to  relate  to  the  provided  rubric 
while  completing  the  performance  assessment  project. 

Students  could  have  scored  more  points  if  they  could  have  connected  the  rubric 
with  the  performance  of  the  PA.  At  the  same  time,  it  has  also  been  noted  that  the  rubric 
can  be  rewritten  to  make  it  little  more  condensed. 

Final 

A majority  of  the  items  on  the  course  final  examination  are  subjective  in  nature, 
while  only  a few  are  objective  (multiple  choices).  Test  analysis  reveals  a much  better 
picture  than  the  midterm.  Eor  the  final,  the  item  difficulty  varied  within  54%  to  81%, 
which  can  be  considered  as  an  acceptable  range.  Also  the  values  of  item  discrimination 
look  pretty  good.  The  lowest  one  is  0.44,  with  most  of  them  lying  around  or  above  0.80. 
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The  only  thing  that  can  be  changed  with  the  test  is  to  include  additional  objective  items  of 
different  types. 

Final  PA 

The  third  PA  was  the  last  PA  for  this  course.  Considering  the  timing,  this  PA 
addresses  all  the  major  topics  that  have  been  covered  within  the  course  and  allows  the 
students  to  exercise  considerable  freedom  (that  means  more  responsibility)  for  the  project 
implementation.  Relative  to  the  PAl  and  PA2,  the  students  have  performed  well  for  the 
PAS.  Also  in  this  case,  students  tried  to  follow  the  rubric  more  when  working  on  the 
PAS. 

Summary  statement 

The  item  analysis  allows  reviewing  the  impact  of  examination  items  on  students’ 
performance,  and  hence  faculty  can  address  the  identified  issues  through  changing  the 
items’  presentation  and  course’s  delivery  aspect  related  to  a specific  item.  At  the  same 
time,  one  can  add  new  item(s)  or  remove  an  existing  item.  Considering  this  as  the  first 
year  of  implementation  of  this  new  course  planning  and  delivery  strategy,  there  are  a 
number  of  things  that  can  be  done  to  make  this  more  effective  and  to  enhance  students’ 
learning  outcomes.  The  enhancement  can  be  done  through  reviewing  the  existing  item 
bank,  developing  new  items,  and  trying  various  teaching  and  learning  models. 

PA  is  a powerful  tool  in  assessing  the  ability  of  a student  in  terms  of  what  one  can 
do  or  perform  after  completing  a course.  Within  the  delivered  course,  three  PAs  were 
implemented  at  various  times  during  the  semester.  It  would  be  useful  to  develop  a PA 
project  bank  for  the  course,  along  with  reviewing/rewriting  the  existing  projects  to 
improve  their  quality.  At  the  same  time,  it  is  important  to  review  the  rubrics  that  have 
been  developed  so  far. 
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Bob  Tatara 


TECH  344  - DIAGNOSTIC  WRITE-UPS 


Midterm  Examination 

To  improve  on  midterm  seoring  and  in  addition  to  leetures  and  homework 
assignments,  the  elass  was  divided  into  two  groups  to  assess  the  difference  between 
cooperative  versus  individual  learning  in  four  content  areas.  The  content  areas  were 
commodity  thermoplastics,  engineered  thermoplastics,  thermosets,  and  elastomers. 
Random  assignment  of  students  to  two  groups  allowed  us  to  assume  the  groups  were 
equivalent.  Each  group  had  approximately  15  students,  while  each  small  learning  group 
was  composed  of  three  students.  The  actual  delivery  of  the  treatment  conditions 
alternated  across  content  areas  and  groups.  An  outline  of  the  experimental  model  is 
provided: 


Table  B.9.d.l: 


Tech  344  - Model  4:  Individual  Learning  vs.  Cooperative  Learning 


Treatment 

Posttest  1 

Posttest  2 

Individual 

Eearning 

Group 

Individual 

Eearning 

Midterm 

10/18/06 

Pinal 

12/11/06 

Cooperative 

Eearning 

Group 

Cooperative 

Eearning 

Midterm 

10/18/06 

Pinal 

12/11/06 

Treatment 

Group 

Content  Area 
I - Commodity 
Thermoplastic 
Study 
Summary 
Questions 

Content  Area 
II  - Engineered 
Thermoplastic 
Study 
Summary 
Questions 

Content  Area 
III  - Thermoset 
Study 
Summary 
Questions 

Content  Area 
IV  - Elastomer 
Study 
Summary 
Questions 

1 

Individual  - 15 

Cooperative 
Groups  #l-#5 

Individual  -15 

Cooperative 
Groups  #l-#5 

2 

Cooperative 
Groups  #l-#5 

Individual  - 15 

Cooperative 
Groups  #l-#5 

Individual  - 15 

It  is  vital  that  any  groups  are  legitimate  cooperative  learning  groups  in  which 
students  are  randomly  assigned  and  outperform  reasonable  expectations  by  their 
combined  efforts.  Additionally,  each  individual  in  the  group  must  be  independently 
evaluated.  Examples  to  accomplish  this  include  keeping  the  group  size  small,  giving 
written  or  oral  examinations  to  students,  and  observing  students  as  they  interact  within 
their  group.  Thus,  each  group  was  limited  to  three  individuals.  Each  group  was  required 
to  complete  an  assignment  demonstrating  knowledge  of  the  content  areas,  and  groups 
were  observed  to  evaluate  learning  interactions.  Group  performance  and  individual 
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performance  within  the  group  were  assessed.  A midterm  examination  was  given  after 
four  group  learning  sessions  corresponding  to  the  four  content  areas. 

From  an  item  analysis  viewpoint,  I did  not  eliminate  any  items  on  the  midterm;  it 
only  had  two  questions  (out  of  30)  considered  for  elimination.  Both  of  these  had  low  Item 
Difficulty  (11%  and  21%)  and  Low  Item  Discrimination  (-0.34  and  0.20).  A closer 
examination  of  the  two  questions  revealed  that  they  were  based  on  reading  assignments 
and  items  not  covered  by  lectures,  student  group  assignments,  or  performance  tasks.  But 
the  items  were  judged  to  be  reasonable;  thus  rather  than  eliminating  them,  I emphasized 
that  the  final  would  also  include  questions  on  the  reading  of  chapters.  (This  includes 
material  not  covered  in  lectures.)  It  was  also  noted  that  students  did  not  perform  better  on 
the  items  from  the  four  content  areas  related  to  the  cooperative  versus  individual, 
traditional  cognitive  learning  activities.  Thus,  this  led  to  another  change  to  include  more 
retention  questions  in  the  final  than  originally  planned.  A total  of  eight  retention 
questions  were  included  in  the  final,  two  from  each  of  the  four  content  areas. 

In  future  courses,  it  would  be  beneficial  to  emphasize  that  students  are  responsible 
for  chapter  readings  as  well  as  lecture  materials.  Also  other  group  learning  models  should 
be  tried. 

There  are  systematic  techniques  available  to  maximize  the  individual’s 
performance  in  a group  setting.  These  techniques  cover  various  ways  of  forming  groups, 
including  ensuring  that  the  groups  are  random  and/or  balanced.  Different  ways  of  group 
functioning  and  dynamic  interaction  are  also  documented.  A sampling  includes  rounds 
where  students  take  turns  speaking;  group  investigation  where  each  group  is  free  to 
choose  a subtopic  within  the  content  area  of  study;  discussions  where  students  take 
opposing  sides  of  an  issue;  and  brainstorming  to  encourage  free-thinking  and  rapid 
development  of  ideas.  Overall,  individuals  should  benefit  from  the  group  learning 
process,  but  this  did  not  occur  presently. 

Final  Examination 

Due  to  rewriting  of  items  based  on  experiences  from  the  midterm,  the  overall 
class  performance  on  the  final  examination  was  10%  better  than  the  midterm.  There  were 
three  questions  (out  of  50)  with  low  Item  Difficulty  (one  at  14%  and  two  at  21%). 
However,  two  of  these  had  reasonable  Item  Discriminations  (0.44  and  0.47),  while  the 
third  was  at  -0.34.  This  third  item  was  discussed  in  lecture  and  included  in  the  textbook 
readings,  so  there  is  no  good  explanation  of  why  the  item  proved  to  be  so  difficult.  Only 
this  item  was  considered  for  elimination.  Three  items  had  Item  Difficulty  scores  of  100%; 
these  items  show  no  discrimination  and  could  also  be  considered  for  dropping.  But,  in  the 
end,  all  items  were  retained  as  a database  to  be  expanded  as  future  exams  give  more 
guidance  of  when  to  eliminate  items.  Certainly,  it  is  expected  that  the  experiences  from 
these  two  examinations  will  provide  for  better  future  test  items. 

There  was  even  better  improvement  on  the  eight  retention  questions;  the  students 
scored  12%  better  on  the  retention  questions  than  on  the  final  as  a whole.  This  indicates 
that  if  special,  or  extra,  attention  is  given  to  critical  topics,  students  are  able  to  perform. 
Different  teaching  and/or  learning  models  ought  to  be  considered  for  such  topics.  This,  in 
conjunction  with  better  group  learning  processes,  should  increase  test  performance, 
including  performance  on  retention  items. 
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Performance  Assessments 

Three  performance  assessment  tasks  were  assigned.  Generally,  scores  were  better 
on  these  than  the  standard  tests,  as  the  tasks  gave  students  an  alternative  opportunity  to 
give  evidence  of  their  knowledge.  The  tasks  also  tested  the  students  at  higher  levels  of 
Bloom’s  taxonomy.  One  of  the  reasons  for  the  success  of  these  was  the  presence  of 
rubrics  that  were  a great  asset  in  the  execution  of  the  performance  assessment  tasks. 
Students  clearly  knew  the  expectations  and  tailored  their  work  to  fulfill  the  requirements. 
This  led  to  higher  scoring,  and  the  better  scores  were  justified. 

The  third  task  included  group  activities.  It  appears  that  students  working  in  groups 
toward  performance  tasks  benefit  more  than  groups  preparing  for  the  midterm  and  final 
examinations.  Students  seem  to  do  better  in  group  settings  where  more  creative,  open- 
ended  projects  are  the  goal.  Future  courses  should  explore  and  exploit  this  trend.  Or  a 
cooperative  versus  individual  learning  study  for  performance  tasks  could  be  conducted. 
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Abijit  Gupta 

Diagnostics  of  Mid  Term  Exam 

1)  Midterm  exam  was  too  long.  Most  students  did  not  have  time  to  attempt  all  problems. 

2)  Items  (8,  13,  15,  24,  33,  36,  and  37),  with  low  discrimination  (items  with  disc  below 
0.1),  and  especially  items  (questions  8,  14,  37),  with  negative  discrimination,  certainly 
cause  concern.  Items  8 and  13  probably  had  low  disc  factor  because  they  were  easy  (item 
8 diff.  of  75%,  item  13  diff  of  91%),  but  they  are  good  questions  and  I may  keep  them  for 
future  tests.  Item  15  was  one  of  the  few  theoretical  questions,  and  students  knew  how  use 
the  formula  without  understanding  the  derivation.  Such  questions  were  not  asked  in 
homework  or  solved  in  class,  and  students  might  not  have  expected  such  question.  It  was 
also  a difficult  item  with  diff  of  18%.  Item  24  should  have  been  a good  item  (item  diff  is 
50%),  but  somehow  it  had  low  disc,  factor.  Maybe  students  were  already  getting  worried 
by  the  time  they  answered  this  question  that  they  would  not  have  enough  time  to  finish 
the  exam.  I am  not  too  surprised  by  low  discrimination  for  items  33,  36,  and  37  because 
the  students  either  did  not  answer  them  or  just  guessed  randomly  due  to  shortage  of  time. 

3)  Regarding  difficulty  of  items,  any  items  being  too  easy  (items  1,  10,  11,  12,  and  13)  or 
too  difficult  (item  4,  15,  21,  and  22)  were  reviewed.  Items  I,  II,  12,  and  13  were  easy, 
and  item  10  probably  was  trivial.  However,  I feel  that  except  item  10,  they  were 
important  questions  and  there  is  no  need  to  remove  them  in  future. 

4)  Since  the  exam  was  too  long,  average  was  low  (41%),  and  it  may  not  have  tested  the 
items  that  were  important  but  were  placed  at  the  tail  end  of  the  exam,  I made  following 
adjustments: 

a)  I gave  an  additional  exam  that  had  only  some  subjective  questions  and  tested 
concepts  that  were  in  the  latter  part  of  the  exam.  Points  obtained  in  this  test  were 
added  to  the  regular  midterm. 

b)  I modified  the  final  exam  so  that  it  can  be  finished  in  time. 

Diagnostics  of  Final  Exam 

1)  Item  13  was  too  easy,  resulting  in  item  diff  of  100%  and  no  discrimination.  I still  do 
not  mind  because  it  is  an  important  concept.  The  rest  of  the  items  seem  to  have 
appropriate  difficulty  index. 

2)  Items  7,11,  and  15  have  negative  discrimination  factor.  Item  7 is  an  important  item 
and  may  need  to  be  reworded  (avoid  double  negative).  Item  1 1 probably  was  due  to  the 
fact  that  during  lab  instruction  the  students  did  not  follow  why  they  had  to  use  the  trigger. 
It  should  be  made  clear  in  next  lab  session.  Item  15  is  a concept  they  should  know  from 
the  midterm  and  may  not  have  remembered  correctly. 

3)  Overall  I am  pleased  with  the  outcome  of  the  exam. 
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Diagnostic  of  Performance  Assessment  1 

My  observation  for  performance  assessment#!  is  as  follows:  PA#1  was  interesting  and 
students  learned  a lot.  The  project  required  vibration/acoustic  measurement  by  every 
student  in  an  industrial  setting.  It  was  difficult  to  find  enough  companies  who  would 
permit  such  measurement  and  even  fewer  companies  have  their  own  measuring 
equipment.  We  have  a limited  number  of  instruments,  so  if  possible,  more  instruments 
should  be  purchased.  Since  the  students  did  not  have  any  background  in  vibration 
measurement  when  they  participated  in  this  exercise,  they  needed  lot  of  help  that  was 
beyond  the  scope  of  the  teaching  assistant.  Fortunately,  I had  a research  assistant  with  lot 
of  experience  in  vibration/acoustic  measurement  and  he  helped  me  out.  However,  such 
help  may  not  always  be  available  and  the  project  accomplishment  may  be  compromised. 

The  mathematical  model  was  difficult  because  this  was  assigned  at  the  beginning  of  the 
semester,  so  it  was  not  a good  item  in  the  rubric. 

Due  to  lack  of  time,  no  in-class  presentation  was  possible  (because  that  would  take  away 
one  class)  and  only  the  report  was  graded.  It  was  not  clear  initially  to  the  students  how 
they  show  their  contribution  to  the  team  and,  therefore,  required  explanation  in  class.  In 
future  this  will  be  more  structured. 

Another  issue  was  that  different  groups  addressed  different  problems  and  had  different 
levels  of  support  from  the  companies  they  visited  and  also  some  problems  were  easier  (or 
more  difficult)  than  the  others.  This  issue  was  addressed  in  PA  #2  and  PA  #3,  where 
every  group  performed  same  projects. 

Diagnostic  of  Performance  Assessment#! 

My  observation  for  performance  assessment  #2  is  as  follows:  PA#2  went  more  smoothly 
than  the  PA#1  because  students  were  dealing  with  a familiar  product.  However,  there 
were  some  small  issues  such  as  students  did  not  understand  just  mentioning  internet  or 
local  shop  visit  is  not  enough  and  they  needed  to  be  more  specific:  URL  for  the  internet 
sites,  name  and  location  of  the  local  stores,  etc.  Students  gave  oral  presentation  for  this 
assignment,  and  I pointed  out  how  to  present  these  better.  One  group  actually  did  more 
than  that  was  asked,  and  I gave  extra  credit  for  that. 

One  problem  I think  was  that  since  half  of  the  class  (experimental  group)  had  to  finish  it 
before  the  midterm,  the  midterm  was  given  somewhat  late  in  the  semester  without  giving 
students  a chance  to  drop  the  course. 

Diagnostic  of  Performance  Assessment  #3 

My  observation  for  performance  assessment  #3  is  as  follows:  For  PA#3,  the  students 
worked  hardest.  That  was  possibly  due  to  many  reasons:  baseball  is  a popular  sport, 
Sammy  Sosa  made  corking  infamous,  any  baseball  player  would  be  interested  to  know 
the  location  of  sweet  spots  and  how  to  choose  a better  bat,  etc.  The  students  also  learned 
some  concepts  (orthogonality,  mass  normalization,  mode  shapes  etc,)  better  due  to  this 
assignment.  Only  one  item  in  the  rubric  (stiffness  matrix  for  the  baseball  bat)  was  too 
difficult,  and  in  future  if  such  question  is  asked,  more  clues  may  be  provided.  During 
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grading,  I realized  that  this  rubric  did  not  ask  specifically  to  show  individual  work. 
Almost  every  group  provided  some  information  that  was  relevant,  but  it  led  to  same  score 
for  every  member  of  the  group.  I will  decide  in  future  whether  that  is  OK  or  if  I should 
require  more  detailed  information  to  ensure  that  everyone  participated  equally. 

Overall  Impression  (all  Performance  Assessments  considered  together) 

Overall,  my  impression  was  that  the  students  performed  better  than  I expected.  They 
worked  very  hard.  Some  said  to  me  that  they  could  do  it  because  only  one  course 
required  these  and  they  would  not  be  able  to  do  these  if  more  than  one  course  required 
them. 

First  assessment  was  probably  given  too  early  and  the  last  one  too  late.  In  the  future,  at 
most  two  Performance  Assessments  should  be  given.  However,  Performance 
Assessments  definitely  augmented  the  learning  of  students. 
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Brianno  Coller 


Comments  on  midterm  concept  test 

Two  of  my  questions  were  flagged  for  a low  discrimination  index. 

Problem  24.  It  appears  that  quite  a few  people  believed  that  this  was  a two-force  member. 
It  is  not.  The  problem  clearly  states  that  the  bracket  has  weight.  The  center  of  gravity 
clearly  appears  on  the  diagram.  I think  it  is  a good  question.  Nonetheless,  I reduced  the 
number  of  possible  points  by  one  for  this  question  - just  because  the  students  needed  it 
and  I was  feeling  charitable. 

Problem  33. 1 am  not  sure  why  this  problem  had  such  a low  index.  This  problem  was  not 
stated  any  differently  than  problems  30,  31,  32,  and  34,  all  of  which  had  moderately  high 
discrimination  indices.  Of  these  five  equilibrium  problems,  however,  it  is  the  one  that  had 
the  lowest  item  average.  Again  because  I was  feeling  charitable,  I reduce  the  total  by  an 
additional  point. 

Comments  on  final  concept  test 

Problem  6.  On  its  surface,  this  looks  like  a quite  simple  problem.  But  because  of  the 
horizontal  forces,  it  is  actually  a bit  harder.  It  takes  some  doing  to  work  it  out  correctly. 
Since  I told  students  they  could  not  use  their  calculators,  this  is  a bit  unfair.  Therefore,  for 
this  problem,  I reduced  the  total  possible  points  by  one.  Doing  so  turns  out  to  be  quite  a 
bonus  for  the  students,  since  the  answer  is  the  same  one  that  you  might  guess  if  you  did 
not  go  through  all  that  extra  calculation. 

Problem  17. 1 do  not  know  why  the  discrimination  index  was  negative  here.  Problem  16 
refers  to  the  same  system  and  its  discrimination  index  is  quite  reasonable.  Therefore,  I 
suspect  the  question  is  written  reasonably  well.  In  my  opinion,  problem  17  is  more 
difficult  and  requires  more  thought  than  problem  16.  Nonetheless,  problem  17  had  a 
significantly  higher  item  average.  So  here  is  my  guess  as  to  what  happened.  Some 
students  reason  through  problems  the  way  we  tell  them  to:  draw  a free  body  diagram  and 
set  up  equilibrium  conditions.  Other  students  rely  on  their  intuition.  Usually,  relying  on 
intuition  without  going  through  the  analysis  steps  is  a bad  idea.  However  in  this  problem, 
it  might  have  been  the  strategy  most  likely  to  succeed. 

Problem  20.  I think  the  discrimination  index  was  low  because  the  item  average  was 
especially  high. 

Problem  28.  One  of  the  students  pointed  out  that  this  question  had  problems  with  it.  I 
need  to  give  the  angles  of  the  string  if  it  is  to  be  determinate.  I reduced  the  number  of 
possible  points  by  1 because  of  this  problem. 

Problems  32,  33.  When  discussing  this  with  my  TA,  these  turned  out  to  be  harder  than  I 
expected.  The  test  statistics  bear  this  out.  I removed  these  two  points. 
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Reflections  on  Performance  Assignment  #l 

Performance  Assignment  #1  was  given  very  early  in  the  semester.  Its  purpose  was  get 
students  involved  with  statics  analysis  early,  using  techniques  amenable  to  students  just 
beginning  the  class,  and  to  motivate  students  for  the  type  of  questions  we  would  analyze 
and  answer  throughout  the  semester. 

One  half  of  the  class  used  a hands-on  experimental  approach  to  determine  string  tensions 
in  a pulley  system.  The  other  half  of  the  class  used  a graphical  force  triangle  approach. 
Anecdotal  feedback  from  the  students  suggests  that  students  enjoyed  applying  their 
analysis  to  a (seemingly)  realistic  situation.  The  biggest  difficulty  with  the  performance 
assignment  appeared  in  the  experimental  group.  The  students  in  this  group  had  a difficult 
time  interpreting  their  experimental  measurements.  Specifically,  many  of  the  students  did 
not  recognize  the  error  in  their  measurements.  They  interpreted  some  of  the  fluctuations 
in  their  measurements  as  something  of  physical  significance,  when  in  reality,  it  was  due 
to  error. 

I can  think  of  three  things  I can  do  to  mitigate  this  in  the  future: 

1.  Discuss  measurement  error  in  the  assignment. 

2.  Buy  cheap  spring  scales  with  which  students  can  more  simply  read  the  forces. 

This  might  be  less  prone  to  error  than  measuring  spring  lengths  and  then 
computing  forces  based  on  these  lengths. 

3.  Make  students  compare  their  results  with  a classmate.  It  is  unlikely  that  two 
students  will  have  the  same  fluctuations. 

Reflections  on  Performance  Assignment  #2 

I really  liked  how  this  assignment  turned  out.  Students  formed  teams  and  had  to  design  a 
cart  to  compete  in  a tug  of  war  competition.  They  had  to  recognize  that  there  are  two 
ways  their  cart  could  lose:  either  by  sliding  or  by  tipping  over.  There  were  firm 
constraints  on  their  design,  and  they  would  have  to  base  their  design  on  anticipating  what 
their  opponent  might  choose  to  do.  Furthermore,  there  was  a bit  of  random  uncertainty 
that  teams  had  to  cope  with. 

The  best  approach  to  designing  the  cart  is  not  something  that  comes  naturally  to  novice 
engineering  thinkers.  Most  students  recognized  both  failure  modes  and  further  noticed 
that  the  two  modes  work  against  each  other.  For  example,  if  one  makes  the  cart  more 
resistant  to  sliding,  it  often  makes  it  more  susceptible  to  tipping  and  vice  versa.  The  best 
option,  therefore,  is  to  design  the  cart  so  that  both  failure  modes  occur  simultaneously. 

Four  teams  (16  students  total)  were  able  to  figure  this  out  on  their  own.  Three  teams  were 
able  to  implement  it  correctly.  One  of  these  three  teams  beat  my  design. 

After  the  competition,  I made  a presentation  in  lecture,  describing  this  design  philosophy 
of  coincident  failure  modes.  Although  a relatively  small  fraction  of  the  students  were  able 
to  figure  it  out  on  their  own  before  the  competition,  almost  all  were  able  to  understand  it 
and  appreciate  it  afterward.  Almost  all  teams  employed  this  strategy  in  Performance 
Assignment  #3. 
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There  is  one  thing  I would  do  differently.  In  Fall  2006, 1 ran  the  competition  (a 
simulation)  at  home  over  the  weekend  . It  would  be  more  exciting  and  more  suspenseful 
to  run  it  in  class. 

Reflections  on  Performance  Assignment  #3 

In  this  exercise  students  had  to  design  and  build  trusses.  The  goal  was  to  support  as  much 
weight  as  possible  while  adhering  to  the  constraints  imposed  upon  them.  The  exercise 
came  in  three  parts.  First,  students  individually  had  to  analyze  a truss  by  hand.  In  part  2, 
they  teamed  up  with  one  other  student  to  write  a Matlab  script  to  perform  the  analysis. 
Finally,  in  Part  3,  they  joined  the  rest  of  their  team  to  perform  the  design. 

In  parts  I and  2, 1 gave  the  students  trusses  with  two  different  topologies  to  analyze.  Both 
trusses  satisfied  the  design  constraints.  Therefore,  in  part  three  of  the  exercise  the  teams 
had  to  figure  out  which  topology  was  better  and  perhaps  explore  a topology  that  is  even 
better  than  the  original  two.  It  turned  out  (not  by  accident)  that  a third  topology  was 
better  than  the  two  given.  Several  teams  figured  this  out.  Nonetheless,  the  teams  that 
stuck  with  the  old  truss  topologies  were  able  to  create  very  good  designs.  On  the  last  day 
of  lecture,  we  broke  the  trusses.  It  was  fun. 

The  next  time  I teach  the  course,  I plan  to  post  targets  for  student  design  teams  to  shoot 
for.  I can  tell  them  how  much  (scaled)  weight  a really  good  truss  should  hold,  so  they  get 
feedback  that  tells  them  how  good  their  design  is. 
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Reinaldo  Moraga 
Analysis  of  Midterm 

The  total  number  of  multiple-choice  items  in  the  midterm  exam  was  25,  from  which  five 
resulted  in  a difficulty  index  below  50%. 

Items  4,  6,  11,  15,  and  22  resulted  in  a discrimination  index  0.37.  After  analyzing  the 
items,  the  conclusion  is  that  they  were  poorly  written  and  some  of  the  distracters  were 
very  similar  to  the  right  answer;  thus  students  might  have  been  a little  confused.  In  all 
these  cases,  a paraphrase  of  the  items  would  be  desirable. 

Analysis  of  Final 

The  total  number  of  multiple-choice  items  in  the  final  exam  was  25,  from  which  eight 
resulted  in  a difficulty  index  below  50%. 

Item  I has  a low  discrimination  index  (0.075),  which  means  extreme  students  got  the 
item  right.  There  is  no  major  problem  with  the  item  itself,  except  that  the  content  is  a 
difficult  one. 

Items  3,  12,  18  and  21  have  decent  discrimination  indexes  (0.33  - 0.44),  which  means  that 
these  items  were  answered  correctly  by  higher  grade  students.  The  correct  choices  from 
these  items  are  very  similar  to  distracters,  which  may  have  confused  some  of  the  students. 

Item  6 was  difficult  and  with  negative  discrimination  index.  This  is  because  the  item  was 
poorly  written,  so  this  item  was  not  a good  one. 

Items  1 1 and  24  have  low  discrimination  index  because  they  required  some  analysis  and 
interpretation  of  given  facts. 

Four  other  items  (5,  8,  10,  and  16)  were  found  to  be  easy  items  but  with  negative 
discrimination  index.  In  some  cases,  higher  grade  students  answered  incorrectly  because 
the  items  were  poorly  written. 

Change  of  Performance  Assessments 

The  Performance  Assessments  (PAs)  for  this  class  were  the  following  ones: 

PA  1:  Research  on  OR  Applications. 

Strength:  This  PA  was  open  in  the  sense  that  students  were  given  a journal  to 
select  one  article  to  discuss.  Higher  Bloom’s  Taxonomy. 

Weakness:  There  were  cases  in  which  students  selected  extremely  difficult  articles 
and  were  overwhelmed  by  them. 

Change:  I think  next  time  I should  select  an  article  and  give  it  to  each  group  to 
analyze. 

2:  Analysis  and  Solution  of  a LP  Case  Study. 

Strength:  This  PA  was  properly  assigned  for  the  students  and  students  were  able 
to  work  well  on  it.  Higher  Bloom’s  Taxonomy. 
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Weakness:  Do  not  see  it  clearly. 

Change:  No  one  for  this  type  of  activity. 

PA  3:  Applying  LP  to  Real-World  Problem. 

Strength:  This  PA  was  open  in  the  sense  that  students  were  supposed  to  find  a real 
application  of  the  topics.  Higher  Bloom’s  Taxonomy. 

Weakness:  Some  teams  came  up  with  similar  cases,  and  even  though  they  worked 
well  on  them,  the  class  did  not  benefit  from  having  a variety  of  cases. 

Change:  Next  time,  I would  assign  a particular  topic  for  students  to  get  a real- 
world  problem  or  I would  monitor  more  carefully  the  cases  studies  they  are 
selecting  from  the  very  beginning. 

Changes  for  the  Courses 

Next  time  I teach  ISYE370, 1 would  implement  cooperative  learning  throughout  all  the 
contents  of  the  course,  and  I would  design  contents  with  more  activities,  taking  advantage 
of  the  students’  learning  styles.  I think  this  is  something  I would  like  to  use  in  some  other 
courses  as  well. 
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Regina  Rahn 


Statement  of  Changes/Concems  Based  in  Item  Analysis:  lENG  475 

After  reviewing  the  item  analysis  for  the  midterm  and  comparing  both  the  item 
discrimination  and  the  item  difficulty  to  the  results  from  Fall  2005,  it  became  apparent 
that  the  results  were  not  satisfactory  for  Fall  2006.  The  main  problems  were  with  the 
discrimination  numbers.  They  were  all  over  the  board.  In  the  fall  of  2005,  when  a test 
was  administered  that  did  not  implement  multiple  choice,  true/false,  etc.,  the  numbers 
were  much  better.  I discussed  this  at  some  length  with  the  students.  We  all  came  to  the 
same  conclusion:  this  format  was  not  conducive  to  success  for  a senior/graduate  level 
course  in  engineering,  where  many  of  the  examples  and  homework  focus  on  multi-step 
implementation  of  problem  solving  techniques. 

The  format  was  changed  for  the  final.  This  time  the  questions  were  still  what  I consider 
to  be  objective,  where  a correct  answer  exists  and  is  attainable.  However,  the  short 
answer  format  was  used  so  more  partial  credit  was  available.  This  resulted  in  somewhat 
improved  numbers  in  the  item  analysis. 

This  is  the  first  time  that  I ever  used  item  analysis.  I believe  that  it  provides  the  instructor 
with  invaluable  information,  allowing  him/her  to  be  able  to  make  improvements 
continually  during  the  semester.  I will  definitely  continue  to  use  this  in  the  future. 
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STUDENT  ASSESSMENT  SUMMARY 
Jule  Dee  Scarborough,  Ph.D. 

(See  Portfolio  Sections  B.9.a,b,c,d,  and  B.9.e.l-5;  also,  A. 5 and  A.7) 

Excerpt  from  Student  Assessment  Summary,  B.9 

Once  the  2005  test  analyses  were  completed  and  we  determined  that  there  were  no  performance 
assessments  in  any  of  the  courses,  the  professors  designed  and  developed  new  midterm  and  final 
exams  for  the  2006  course.  Using  the  results  of  those  analyses  diagnostically  and  the  new  2006  student 
learning  outcomes,  the  professors  developed  a Table  of  Specifications  to  guide  their  creation  of  a new 
midterm  and  final  exam  for  the  2006  course.  They  each  developed  an  objective  test  item  bank  of 
multiple  questions  for  each  student  learning  outcome.  Once  the  objective  test  items  were  developed, 
they  chose  items  for  each  exam,  midterm  and  final,  and  assembled  the  tests.  If  they  preferred  to  include 
problems  to  solve,  those  were  added  as  well.  The  program  leaders  provided  feedback  throughout  the 
entire  analysis  and  development  process.  To  further  ensure  that  the  test  items  and  assembled  tests 
actually  measured  knowledge  or  skills  inherent  in  an  outcome,  the  professors  mapped  the  outcomes  to 
the  corresponding  tests  and  specific  items.  This  helped  them  realize  where  they  needed  more  items  or  a 
different  type  of  item  and,  especially,  where  there  were  gaps  in  the  measurement  of  critical  outcomes. 
(See  worksheet  below  and  other  examples  in  B.9.2.e.l-5.)  Although  the  tests  were  not  perfect,  they 
were  greatly  improved.  (See  Portfolio  Section  B.9. a for  the  comparison  for  differences  between  the 
2005  and  2006  tests;  see  Program  Description,  Portfolio  Section  A.7,  for  further  information  on  the 
Test  Analysis  and  Development  program  components.) 

Faculty  examples  of  their  course  outcomes  to  test  and  test  item  analysis  are  copied  below. 
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B.9.e.l:  IENG370  Operations  Research  - R.  Moraga  (Explanation  in  B.9) 


Student  Learning  Objectives  and  Outcomes 

Assessments:  Test  Alignments 
Midterm  & Final 

Student  Learning  Objectives/Outcomes- 
Major 

student  learning  objectives  - 
minor 

Corresponding  Tests  and 
Test  Items 

1 

1.  To  apply  fundamental  methods  of 
deterministic  operations  research  models  to  solve 
industrial  engineering  problems  (related  to 
planning,  scheduling,  budgeting,  etc.) 

1.1.  To  apply  linear  programming  (LP) 
models 

a 

To  formulate  LP  models 

Examl,MCl-4/SA17-20; 
Final,  MC1-3/MTF17-20 

b 

To  solve  a LP  model  by 
applying  the  Simplex 
Algorithm  (SA) 

Fxaml,  MC5-8; 
Final,  MC4,5 

c 

To  analyze  and  interpret 
results  by  applying  sensitivity 
analysis 

Fxaml,  MC9-16 
Fxam2 

d 

To  examine  special  cases  of 
LP  models  (transportation 
and  networks)  and  to  solve 
them  using  modified  SA 
versions 

Fxam2 

2 

1.  To  apply  fundamental  methods  of 
deterministic  operations  research  models  to  solve 
industrial  engineering  problems  (related  to 
planning,  scheduling,  budgeting,  etc.) 

1.2.  To  apply  integer  programming  (IP) 
models  to  solve  industrial  problems. 

a 

To  formulate  IP  models 

Fxam2 

Final,  MC8-1 1/SA13-16 

b 

To  solve  IP  models  by  using 
the  branch  and  bound 
algorithm  (BBA) 

3 

1.  To  apply  fundamental  methods  of 
deterministic  operations  research  models  to  solve 
industrial  engineering  problems  (related  to 
planning,  scheduling,  budgeting,  etc.) 

1.3.  To  acquire  basic  knowledge  of  dynamic 
programming  (DP)  modeling  to  solve  LP 
models. 

a 

To  reformulate  a LP  model 
using  DP  modeling. 

Final,  MC6,12 

b 

To  apply  the  dynamic 
programming  recursive 
approach. 

4 

2.  Students  will  apply  the  appropriate  LP  model 
to  solve  a real-world  problem. 

2.1.  To  choose  a real-world  problem  from  the 
following  sector:  manufacturing,  services, 
banking,  transportations,  educational,  or 
health  care. 

Project 

5 

2.  Students  will  apply  the  appropriate  LP  model 
to  solve  a real-world  problem. 

2.2.  To  define  the  problem 

a 

To  describe  context  situation. 

Project 

b 

To  identify  at  least  10 
decision  variables  for  the 
problem 

c 

To  identify  at  least  10 
constraints  for  the  problem 

6 

2.  Students  will  apply  the  appropriate  LP  model 
to  solve  a real-world  problem. 

2.3.  To  formulate  the  problem  using  a LP 
model 

a 

To  apply  assumptions  of  LP 
models 

Project 

b 

To  define  decision  variables 
and  technological  coefficients 

c 

To  define  and  construct  a 
performance  criterion 
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d 

To  construct  the  region  of 
feasible  solutions 

7 

2.  Students  will  apply  the  appropriate  LP  model 
to  solve  a real-world  problem. 

2.4.  To  solve  the  problem  using  SA 

a 

To  reduce  any  LP  model  to 
the  standard  form 

Project 

b 

To  apply  steps  of  the  SA  and 
its  fundamental  algebra 

8 

2.  Students  will  apply  the  appropriate  LP  model 
to  solve  a real-world  problem. 

2.5.  To  analyze  and  interpret  results 

Project 

9 

2.  Students  will  apply  the  appropriate  LP  model 
to  solve  a real-world  problem. 

2.6.  To  communicate  results  in  a manner  that 
unites  theory,  reasoning,  analysis,  and 
criticism  in  speaking  and  writing. 

a 

To  write  a technical  report 

Project 

b 

To  present  the  report 

This  class  has  three  exams.  MC:  Multiple  Choice;  SA:  Short  Answers;  MTF:  Multiple  True-False 
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B.9.e.2;  Course  Title  and  Number  (Explanation  in  B.9)  Abul  Azad,  Technology  277 


Student  Learning  Objectives  and  Outcomes 

Assessments:  Test  Alignments 
Midterm  & Final 

Student  Learning  Objectives/Outcomes- 
Major 

student  learning  objectives  - minor 

Corresponding  Tests  and 
Test  Items 

1. 

To  examine  the  components  of  a digital 
system. 

a 

To  contrast  between  analog  and  digital 
signals 

Multiple  choice:  laAl.  IbCl,  IcCl. 
lcC2,  IdKl,  ldK2,  leCl 

b 

To  classify  binary  digits,  logic  levels, 
and  digital  waveforms 

c 

To  compare  basic  logic  operations 

d 

To  categorize  fixed  function  integrated 
circuits 

e 

To  interpret  the  operation  of  simple 
digital  systems 

2. 

To  examine  the  structures  for  various  number 
systems. 

a 

To  distinguish  between  various  parts  of 
number  systems. 

Multiole  choice:  2aCl.  2aAl.  2bCl. 
2bC2 

Short  answer:  2aKl.  2aK2.  2aK3.  2aCl 

b 

To  examine  the  counting  in  binary, 
octal,  decimal,  and  octal. 

3. 

To  distinguish  the  conversion  methods  for 
various  number  systems. 

a 

To  convert  between  binary  and  decimal 

Multiole  choice:  3aAl.  3aA2.  3aA3. 
3aA4,  3aA5,  3bAl,  3bA2,  3bA3,  3cAl, 
3cA2 

Short  answer:  3cCl.  3bA2,  3bA3.  3cAl. 
3cA2 

b 

To  convert  between  binary  and 
hexadecimal 

c 

To  convert  between  binary  and  octal 

4. 

To  perform  different  binary  arithmetic 
operations: 

addition,  subtraction,  Ts  complement,  2's 
complement,  and  signed  numbers. 

a 

To  examine  the  basic  rules  involving 
each  of  the  operations. 

Multiole  choice:  4aCl.  4aC2.  4aC3. 
4bAl,4bA2,  4bA3,4bA4. 

Short  answer:  4bAl.  4b A2.  4b A3.  4bA4. 
4aKl 

b 

To  use  the  rules  to  perform  each  of  the 
operations. 

5. 

To  examine  the  operation  and  use  of  various 
logic  gates  with  different  input  patterns:  AND, 
OR,  and  NOT,  NAND,  NOR,  XOR  and 
XNOR. 

a 

To  develop  the  truth  tables  of  various 
logic  gates  using  established  rules. 

Multiole  choice:  5bCl.  5bC2.  5cAl. 
5dAl,  5dA2,  5cA2,  5dA4,  5dA5. 

Short  answer:  5bCl,  5bC2,  5bC3,  5bC4. 
5bC5,  5bC6,  5dCl,5dC2 

b 

To  use  the  truth  tables  to  identify  output 
pattern  of  a logic  gate  for  a given  set  of 
input. 

c 

To  predict  output  logic  levels  for  a 
pulse  input  pattern. 

d 

To  recommend  the  use  of  appropriate 
logic  gate(s)  for  a given  application. 

6. 

To  analyze  the  properties  of  fixed-function 
logic  integrated  circuits  (IC):  Complementary 
Metal  Oxide  Semiconductor  (CMOS)  and 
Transistor-Transistor  Logic  (TTL). 

a 

To  identify  various  supply  voltage  and 
power  requirements  for  CMOS  and 
TTL  ICs. 

Multiole  choice:  6aAl.  6aA2.  6b A2. 
6bA3,  6cAl,  6dCl,  6fCl,  6fC2 
Short:  6aKl,  6aK2,  6aK3,  6aK4,  6bKl, 
6bK2,  6bK3,  6bK4,  6cKl,  6cK2,  6eCl, 
6eC2,  6fKl,  6fK2,  6fK3,  6fK4,  6fK5, 
6fK6,  6fK7,  6fK8. 

b 

To  analyze  the  generic  numbering 
convention  for  CMOS  and  TTL  ICs. 

c 

To  classify  common  logic  gate  ICs 
according  to  their  standard  identifier 
digit. 

d 

To  examine  the  logic  gate  configuration 
within  an  IC. 

e 

To  compare  alternative  logic  symbols 
for  representing  logic  gates  while 
drawing  a circuit  diagram. 

f 

To  examine  the  voltage  values  for  input 
output  logic  levels  for  CMOS  and  TTL 
ICs. 
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7. 

To  analyze  the  performance  characteristics 
and  parameters  for  logic  gates  and  evaluate 
their  significance  in  digital  design. 

a 

To  estimate  the  propagation  delay  for  a 
given  logic  gate  and  realize  its 
significance  in  digital  design. 

Multiple  choice:  7aAl.  7aCl.  7bAl. 
7cAl,7dCl 

Short  answer:  7aCl.  7aAl,  7bA2.  7cAl. 
7dCl 

b 

To  estimate  the  speed-power  product  as 
a measure  of  the  performance  of  a logic 
circuit. 

c 

To  estimate  fan-out  and  loading 
conditions  while  designing  a logic 
circuit. 

d 

To  interpret  data  sheets  for  different 
logic  gate  ICs. 

e 

To  evaluate  data  sheet  information 
while  making  design  decisions. 

8. 

To  use  Laws  and  Rules  of  Boolean  algebra 
and  DeMorgan’s  Theorems  for  manipulating 
Boolean  expressions. 

a 

To  use  the  commutative,  associative, 
and  distributive  laws  to  manipulate 
Boolean  expressions. 

Multiole  choice:8aKl.  8aCl.  8bCl. 
8cAl,8cA2 

Short  answer:  8aKl.  8aK2.  8aK3.  8aK4. 
8aK5 

b 

To  examine  the  use  of  Boolean  rules 
while  manipulating  Boolean 
expressions. 

c 

To  use  DeMorgan’s  Theorems  for 
manipulating  Boolean  expressions. 

d 

To  adapt  the  Boolean  laws.  Boolean 
rules,  and  DeMorgan’s  Theorems  while 
minimizing  Boolean  expressions. 

9. 

To  analyze  digital  logic  circuits  using  Boolean 
algebra. 

a 

To  develop  a Boolean  expression  for  a 
given  logic  circuit. 

Multiole  choice:  9aAl.  9aA2.  9cAl. 
9cA2 

Short  answer:  9aCl.  9aC2 

b 

To  evaluate  a Boolean  expression  and 
prepare  a truth-table  for  the  logic 
circuit. 

c 

To  demonstrate  the  use  of  Boolean 
algebra  while  minimizing  Boolean 
expressions. 

10. 

To  develop  and  analyze  standard  forms  of 
Boolean  expressions:  Sum-of-Products  (SOP) 
and  Product-of-Sums  (POS). 

a 

To  manipulate  Boolean  expressions  to 
form  SOP  and  POS. 

Multiole  choice:  lOaCl.  10aC2.  10aC3. 
lObAl,  10bA2 

b 

To  implement  SOP  and  POS 
expressions  using  available  logic  gates. 

c 

To  convert  Boolean  expression  between 
Standard  SOP  and  POS  forms. 

11. 

To  evaluate  the  relationship  between  truth 
tables  and  standard  forms  of  Boolean 
expressions  (SOP  and  POS). 

a 

To  transform  SOP  expression  to  truth 
table  format. 

Multiole  choice:  llcAl,  llcA2.  lldAl. 
lleAl 

Short  answer:  llcCl,  tide!,  lleCl 

b 

To  transform  POS  expressions  to  truth 
table  format. 

c 

To  develop  standard  form  of 
expressions  from  a truth  table. 

d 

To  convert  non-standard  forms  SOP 
forms  to  standard  SOP  form 

e 

To  convert  non-standard  forms  POS 
forms  to  standard  POS  form 

12. 

To  minimize  logic  expressions  using 
Karnaugh  map  (K-map). 

a 

To  develop  K-maps  with  different  size 
of  input  variables  (1  to  4). 

Multiole  choice:  12cAl,  12cA2 
Short:  12bCl,  12bC2,  12cAl,  12cA2, 
12dAl 

b 

To  map  SOP  expressions  on  K-maps. 

c 

To  develop  minimized  expressions 
from  K-maps. 

To  construct  K-map  from  a non- 
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d 

structured  SOP  expression. 

13. 

To  analyze  digital  systems  using 
combinational  logic. 

a 

To  design  a combinational  logic  system 
for  a given  problem. 

Multiole  choice:  13fAl.  13fA2 
Short  answer:  13fAl 

b 

To  design  a logic  circuit  using  standard 
logic  gates  from  a given  Boolean 
expression. 

c 

To  design  a logic  circuit  from  a truth 
table. 

d 

To  design  logic  circuit  only  with 
NAND  or  NOR  gates. 

e 

To  analyze  the  operation  of  a 
combinational  logic  circuit  with  pulse 
inputs. 

f 

To  develop  Boolean  expression  from  a 
given  logic  circuit 

14. 

To  evaluate  combinational  logic  circuits  for 
commonly  used  digital  functionalities:  Half- 
adders and  full-adders,  parallel  binary  adders, 
comparators,  BCD  to  decimal  decodes,  BCD 
to  7-segment  decoders,  encoders,  multiplexers, 
and  demultiplexers. 

a 

To  examine  their  design  principles 

Multinle  choice:  14aCl.  14aC2.  14aAl. 
14aA2,  14aC3,  14aC4,  14cCl,  14dKl, 

b 

To  develop  combinational  logic  circuits 
using  commercially  available  ICs  to 
implement  these  common  digital 
functionalities. 

14dAl,  14dK2,  14fKl,  14dK3,  14fK2, 
14fK3,  14fK4,  14fK2,  14iKl,  14gK3, 
14gK4,  14gK5,  14jKl,  14jK2 
Short  answer:  14aKl.  14aK2.  14bK2. 
14dC2,  14fKl,  14gKl,  14iAl,  14jKl, 

c 

To  analyze  the  design  of  a magnitude 
comparator. 

14jKl 

d 

To  analyze  the  function  of  a decoder. 

e 

To  design  and  develop  higher  size 
decoder  using  smaller  size  decoder  ICs. 

f 

To  explain  the  use  of  BCD-7-Segment 
decoder  for  a real-life  application. 

g 

To  explain  the  design  of  encoders  using 
commercial  ICs. 

h 

To  analyze  the  function  of  an  encoder. 

* 

To  evaluate  the  operation  of 
multiplexers  and  their  implementation 
using  commercially  available  ICs. 

j 

To  evaluate  the  operation  of 
demultiplexers  and  their 
implementation  using  commercially 
available  ICs. 

15. 

To  evaluate  the  properties  of  Latches,  Flip- 
Flops,  and  timers. 

a 

To  contrast  between  Latches  and  Flip- 
Flops. 

Multiple  choice:  15aKl,  15bKl.  15cKl, 
15dKl,  15eCl,  15gKl 
Short  answer:  15aKl.  15bCl.  15bC2. 
15bC3,  15bC4,  15cCl,  15cC2,  15dCl, 
15dC2,  ISeAl 

b 

To  evaluate  the  properties  of  edge- 
triggered  J-K  Flip-Flop. 

c 

To  evaluate  the  properties  of  edge- 
triggered  D Flip-Flop. 

d 

To  evaluate  the  properties  of  edge- 
triggered  S-R  Flip-Flop. 

e 

To  utilize  the  asynchronous  Preset  and 
Clear  inputs  of  Flip-Flops. 

f 

To  examine  the  operating 
characteristics  of  Flip-flops,  such  as- 
propagation  delay  times,  set-up  time, 
hold  time.  Maximum  clock  frequency. 
Pulse  width,  and  Power  dissipation. 

g 

To  compare  the  properties  of 
commercially  available  Flip-Flop  ICs. 
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16. 

To  examine  the  use  of  Flip-Flops  in  practical 
applications. 

a 

To  design  parallel  data  storage  using 
Flip-Flops. 

Multiole  choice:  16aKl.  16bKl.  16aAl. 
16aA2,  16cCl,  16cC2,  16cAl,  16cA2 
Short  answer:  16bAl.  16cAl 

b 

To  implement  frequency  division  using 
Flip-Flops. 

c 

To  design  binary  counter  using  Flip- 
Flops. 

17. 

To  design  applications  using  the  555  Timer 

a 

To  use  555  timer  for  monostable 
operation. 

Multinle  choice:  17cCl.  17bCl.  17cCl. 
IVcAl,  17bAl,  17aAl 

b 

To  use  555  timer  for  bistable  operation. 

c 

To  use  555  timer  for  astable  operation. 

18. 

To  design  and  study  of  counter  applications 
using  Flip-Flops. 

a 

To  design  and  analyze  asynchronous 
binary  counters. 

Multinle  choice:  ISaCl.  IScCl,  ISaAl. 
18aA2 

b 

To  design  and  analyze  asynchronous 
decade  counter. 

c 

To  design  and  analyze  synchronous 
binary  counter. 

d 

To  design  and  analyze  synchronous 
BCD  decade  counter 

e 

To  design  and  analyze  up/down 
synchronous  counter 

19. 

To  design  and  study  of  various  shift  register 
applications. 

a 

To  demonstrate  the  use  of  D Flip-Flop 
as  a shift  register. 

Multinle  choice:  19aCl.  19bCl.  19cAl 

b 

To  design  and  develop  serial  In/  serial 
Out  shift  register. 

c 

To  design  and  develop  serial  In/  parallel 
Out  shift  register. 

d 

To  design  and  develop  parallel  In/  serial 
Out  shift  register. 

e 

To  design  and  develop  parallel  In/ 
parallel  Out  shift  register. 

f 

To  design  and  develop  bi-directional 
shift  register. 
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B.9.6.3:  IENG  475  - Decision  Analysis  - Regina  Rahn  (Explanation  in  B.9) 


Student  Learning  Objectives  and  Outcomes 

Assessments:  Test 
Alignments 
Midterm  & Final 

Student  Learning  Objectives/Outcomes- 
Major 

Student  Learning  Objectives  - minor 

Corresponding 
Tests  and 
Test  Items 

I 

To  learn  to  use  a specific  set  of  analytical  tools  for 
technical  decision  making  under  uncertainty. 

A 

Students  will  be  able  to  construct/create  a 
decision  tree  to  aid  in  determining  the  best 
course  of  action  for  a given  set  of 
circumstances 

1 . To  define  the  states  of  nature  of  the 
system,  process,  or  situation 

2.  To  develop  the  branch  structure  of 
the  tree 

a.  To  identify  decision  nodes; 
what  are  the  items  the 
decision  maker  chooses 

b.  To  identify  the  chance  nodes; 
the  events  that  occur  by 
chance  with  a given 
probability 

c.  To  draw  the  arcs,  which 
define  the  sequences  and 
relationships  between  nodes 

3.  To  identify  the  outcomes 

a.  To  define  the  choices  for  a 
decision  node 

b.  To  define  the  possible 
outcomes  of  a chance  node, 
which  are  a set  of  mutually 
exclusive  outcomes 

c.  To  define  the  “consequence,” 
or  the  final  outcome  of  a 
branch 

4.  To  solve  for  the  expected  value  of 
the  decision  tree  (EV,  EMV) 

a.  To  construct  the  joint, 
conditional,  and  marginal 
probabilities 

b.  To  calculate  all  branch 
probabilities  of  the  tree 

i.  To  apply  Baye’s  Theorem 

ii.  To  implement  the  inverse 
tree  structure  technique 

5.  To  find  and  compare  the  expected 

value  of  both  sample  and  perfect 
information  (EVPI,  EVSI) 

a.  To  construct  the  decision 
trees  to  calculate  EVPI  and 
EVSI 

b.  To  evaluate  the  relevance 
and  importance  of  the  values 
obtained  for  EVPI  and  EVSI 
to  the  decision  process 

HW  #1  (A) 

HW  #2  (A) 

HW  #3  (B,C) 

HW  #4  (D) 

HW  #5  (E) 

Performance  Task  #1 
(A) 

Midterm  #1-18  (A) 

Midterm#  19-2 1(B) 

Midterm  # 22-30  (C) 

Pinal  # 19-22  (A) 

Pinal  #6-10,  23-25 
(C) 

Pinal#  1-5  (D) 
Pinal#  11-18  (E) 
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B 

Students  will  be  able  to  construct  the 
formulae  for  conditional  likelihood  ratios 
and  to  calculate  the  probabilities/ratios 

1.  To  calculate  the  conditional 
likelihood  ratio  (CLR)  associated 
with  a particular  observation 

2.  To  calculate  the  CLR  for  multiple 
observations 

3.  To  compare  these  results  with 
B aye’s  Theorem 

C 

Students  will  be  able  to  construct  a single- 
attribute utility  function 

1 . To  propose  a lottery  that  would  be 
appropriate  for  evaluation  of  risk 

2.  To  calculate  the  necessary  values 
from  the  lotteries  needed  for 
analysis 

a.  To  calculate  the  certainty 
equivalent 

b.  To  calculate  the  risk 
premium 

c.  To  calculate  the  selling  price 

d.  To  calculate  the  buying  price 

e.  To  calculate  the  insurance 
premium 

3.  To  translate  the  lotteries  into  a 
mathematical  function 

4.  To  create  a graphical  interpretation 
of  the  function 

5.  To  analyze  and  compare  two 
lotteries  at  a time,  to  be  used  when  a 
reference  point  is  needed 

D 

Students  will  be  able  to  develop  and 
analyze  fault  trees 

1 . To  describe  the  events  of  a tree 
for  a given  scenario 

a.  To  identify  the  top  event 

b.  To  define  primary  and 
secondary  failures  and 
command  faults 

c.  To  identify  the  sequence  of 
events 

2.  To  create  the  fault  tree  for  a given 
scenario  (such  as  the  safety 
analysis  of  a system)  using 
deductive  analysis 

a.  To  define  the  “and”  and  “or” 
gates 

b.  To  implement  the  logic 
symbols  into  the  tree 

3.  To  develop  and  analyze  dual  fault 
trees 

a.  To  translate  the  meaning  of  a 
system  failure  into  the 
reliability  of  the  system 
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E 

Students  will  be  able  to  construct  a multi- 
attribute utility  function 

1 . To  construct  a graphical  model 
for  the  function 

2.  To  develop  a mathematical  model 

for  the  function 

II 

To  apply  analytic  decision  making  techniques  for 
technical  decision  making  under  uncertainty  and  to 
analyze  and  evaluate  the  results. 

A 

Students  will  be  able  to  perform  sensitivity 
analysis  on  a decision  tree  scenario 

1 . To  construct  a graph  to  interpret 
the  results 

a.  To  plot  the  two  dimensional 
plane  for  a single  chance 
node  sequence 

i.  To  interpret  the  graph 

b.  To  plot  the  plane  for  two 
sequential  chance  events 
(plot  the  pq  plane) 

i.  To  interpret  the  graph 

2.  To  determine  the  threshold 
probability  levels 

a.  To  decide  if  the  current 
solution  is  a good  one  and 
justify  the  decision 

b.  To  determine  the  salient 
factors  (variables)  in  the 
decision  being  modeled  by 
the  current  tree 

c.  To  make  recommendations 
for  improvements 

HW  #1  (A) 

HW  #2  (A) 

HW  #3  (B,C) 

HW  #4  (D) 

HW  #5  (E) 

Performance  Task  #2 

Midterm  # 1-18  (A) 

Midterm#  19-2 1(B) 

Midterm  # 22-30  (C) 

Final  # 19-22  (A) 

Final  #6-10,  23-25 
(C) 

Final#  1-5  (D) 
Final#  11-18  (E) 

B 

Students  will  be  able  to  make  inferences 
about  a system  based  on  the  values  of  the 
conditional  likelihood  ratios 

1 . To  determine  the  pass  or  fail 
rates  allowed  for  a given  set  of 
specification  limit 

2.  To  determine  probabilities  of 
occurrence  for  multi-variate 
systems  based  on  the  values  of 
the  CLR’s 
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c 

Students  will  be  able  to  evaluate  a given 
utility  function 

1.  To  analyze  the  degree  of  risk 
aversion  from  the  utility  function; 
risk  prone,  risk  averse,  risk 
neutral 

2.  To  assess  the  process  being 
modeled;  is  the  model  sufficient 
a.  To  determine  if  the  model 

needs  to  have  more  attributes 

3.  To  recommend  any  changes  in 
the  model 

a.  To  decide  if  more  iterations 
are  necessary  for  the  lotteries 

b.  To  decide  if  any  of  the 
lotteries  need  to  be 
referenced 

c.  To  determine  if  the  utility 
function  is  consistent  with 
the  behavior  of  the  decision 
maker 

D 

Students  will  be  able  to  evaluate  the 
scenarios  modeled  by  a fault  tree 

1 . To  determine  the  reliability  and 
failure  probability  relationships 

2.  To  propagate  the  probabilities 
through  the  gates 

3.  To  perform  a qualitative 
evaluation 

a.  To  determine  cut  sets,  as  well 
as  the  minimal  cut  set 

4.  To  perform  a quantitative 
evaluation 

a.  To  determine  cut  sets,  as  well 
as  the  minimal  cut  set 

b.  To  develop  the  equivalent 
fault  tree 

c.  To  obtain  the  numerical 
probability  that  a given  cut 
set  induces  failure  of  the 
system 

5.  To  implement  the  additive 
model  for  multi-attribute 
scenarios 

E 

Students  will  be  able  to  implement  the 
additive  model  for  multi-attribute  utility 
theory 

1 . To  assess  a two-attribute  utility 
function 

2.  To  determine  the  weights  of  the 
functions  (the  k’ s) 
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Ill 


A 


To  structure  and  solve  complicated  decision  problems 

~C 

~D 


E 


Students  will  be  able  to  define  a scenario  to 
be  analyzed  with  the  decision  analysis 
techniques 

1 . To  research  an  area  in  which 
the  tools  can  be  applied,  and  to 
choose  a problem  for  study 

2.  To  define  the  scope  for  the 
problem  chosen 

3.  To  determine  a set  of 
objectives  for  the  given 

problem 

Students  will  be  able  to  formulate/design  a 
possible  solution  approach  to  the  given 
problem 

1 . To  determine  an  appropriate 
solution  technique  to  be  applied 
to  the  problem  from  the  set  of 
available  tools 

2.  To  formulate  the  solution 
Students  will  be  able  to  implement  the 
solution  techniques  to  obtain  a first  round 
solution 

1 . To  solve  the  formulation 

Students  will  be  able  to  analyze  the  solution 
found  in  order  to  assess  the  current  state 

1.  To  assess/analyze  the  value  of 
the  solution  found  (does  it 
make  sense  numerically?) 

2.  To  evaluate  the  significance  of 
the  solution  (what  does  this 
mean  for  the  decision  maker?) 

Students  will  be  able  to  recommend  a 
course  of  action  based  on  the  original 
solution 

1 . To  establish  if  the  decision  maker 
is  risk  averse,  risk  prone,  or  risk 
neutral 

2.  To  recommend  the  path  that  the 
decision  maker  should  embark 

upon,  based  on  the  results 


HW#2 


Performance  Task  #2 


Performance  task  #3 


F 

Students  will  be  able  to  assess  the 
usefulness  of  the  solution  and  recommend 
any  changes  in  the  process 

1 . To  evaluate  if  the  current  solution 
adequately  answers  the  most 
important  questions  facing  the 
decision  maker 

2.  To  analyze  the  solution  for 
computational  accuracy 

3.  To  decide  if  the  solution  should  be 
improved  upon 

a.  To  conclude  if  factors 
(attributes)  need  to  be  added 

b.  To  conclude  if  factors 
(attributes)  need  to  be 
removed 

c.  To  determine  if  the  correct 
probabilities  were  utilized 

IV 

To  identify  and  define  any  limitations  of  the  models 
and  techniques  for  rational  decision-making. 

Performance  Task  #2 
Performance  Task  #3 

V 

To  demonstrate  an  ability  to  effectively  present  the 
problem,  solution,  and  recommendations  of  a 
complicated  decision  scenario 

A 

Students  will  explain  results  via  a formal 
presentation 

Performance  Task  #2 
Performance  Task  #3 

B 

Students  will  explain  results  via  a formal 
written  technical  report 
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B.9.C.4:  Worksheet  Objectives  and  Tests-  Mayl5_MEE321  (Explanation  in  B.9) 


MEE  321  Mechanical  Vibrations  I 


Student  Learning  Objectives  and  Outcomes 

Assessments:  Test  Alignments 
Midterm  & Final 

Student  Learning  Objectives/Outcomes- 
Major 

Student  Learning  objectives  - 
minor 

Corresponding  Tests  and 
Test  Items 

Midterm  abbreviated  as  M 
Pinal  abbreviated  as  P 

1 

A.  Classify  types  of  Vibration 

a 

Discuss  common  vibration 
phenomenon 

M-1,  2 

i)  Identify  if  vibration  is 
Deterministic  and  classify 
as  ajSinusoidal,  b) 
Periodic,  and  c)  Transient 

ii)  Identify  if  vibration  is 
Random  and  classify  as  a) 
Stationary  and  b)  Non 
Stationary 

b 

Identify  the  source  of 
excitation  as  i)  Eree  or 
iijEorced  Vibration 

c 

Identify  possible  source  of 
energy  loss  and  classify  as 

i)  Undamped  or 

ii)  Damped 

d 

Classify  the  system  as  i) 
linear  or  ii)  nonlinear 

B.  Solve  Kinematics  and  Kinetics  problems 
involving  particle  and  rigid  body  analyses. 

a 

b 

Define  degrees  of  Ereedom 

Solve  Particle  Kinematics 
problems  for  i)  Velocity 
analysis,  and  ii) 
Acceleration  analysis 

M-3,  M-4,M-20,  M-30 
M-5 

c 

Solve  Rigid  Body 
Kinematics  problems  for 
i)  Relative  Velocity 
analysis  ii)  Relative 
Acceleration  analysis 

M-6,  M-9,  M-32 

Decide  how  to  choose 

M-6, 
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d 

Particle  and/or  Rigid  Body 
formulation 

Solve  the  kinetics  problem 

M-6,  M-7,  M-8,  M-9 

e 

i)  Identify  method  of 
solution  by  identifying  list 
of  variables  ii)  Draw  FBD 
and  MAD  to  solve  for 
instantaneous 
forces/accelerations 

M-21,M-22,  M-31,M-32 

C.  Convert  a real  life  vibration  to  a 

i)  Convert  a complex 

M-10 

mathematical  statement 

a 

system  to  simple  sub- 
systems ii)  Draw  the 
Schematic  of  the  sub- 

systems 

M-11 

Compute  equivalent 

D.  Identify  basic  elements  used  to  solve 

stiffness  for  i)  springs  in 

M-12,  M-13,  M-36 

vibration  problem. 

a 

series  ii)  springs  in  parallel, 
or  iii)  combined  effect  in  a 
SDOF  system 

Compute  equivalent  mass 

b 

Define  basic  vibration 

E.  Analyze  vibration  of  a system  subjected 

terminology  for  sinusoidal 

M-14 

harmonic  and  periodic  motion 

a 

motion 

i)  Obtain  Fourier  series 
expansion  for  periodic 

M-15,M-16 

b 

motion 

ii)  Reconstruct  a periodic 
wave  from  first  few 
harmonics 

a 

Identify  the  degree  of 

2 

A.  Solve  for  free  undamped  vibration  of  a 

freedom 

M-17,  M-20 

Single  Degree  of  Freedom  System 

b 

i)  Derive  equation  of 
motion  for  undamped 
translation  system 

M-17,M-18 

ii)  Solve  the  differential 
equation  of  motion  and 
compute  natural  frequency 

M-19 

i)  Derive  equation  of 
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c 

motion  for  undamped 
rotational  system 

M-17,  M-18 

ii)  Solve  the  differential 
equation  of  motion  and 
compute  natural  frequency 

M-19,  M-23 

B.  Apply  modeling  of  free  vibration  of 
undamped  System 

a 

i)  Use  compound 
pendulum  for  solving 
moment  of  inertia 

ii)  Compute  center  of 
percussion  and  use  it  for 
sports  applications 

C.  Check  system  stability 

a 

Compute  equivalent  mass 
and  system  and  check  if 
system  is  stable 

3 

A.  Solve  for  free  vibration  of  a viscously 
damped  Single  Degree  of  Freedom  System 

a 

Derive  equation  of  motion 
of  a viscously  damped 
SDOF  system 

M-20,  M-21,  M-22,  M-23, 
M-24,  M-33 

b 

Compute  Critical  Damping 
Constant  and  Damping 
ratio 

M-24,  M-34 

c 

Solve  for  the  response  of 

i)  underdamped, 

ii) critically  damped  and  iii) 
overdamped  system  due  to 
given  initial  conditions 

d 

Compare  the  undamped 
and  damped  natural 
frequencies  and  understand 
its  relevance  in  terms  of 
comparison  of  theory  and 
experiment 

M-25,  M-26,  M-27,  M-28, 
M-29 

B.  Solve  for  free  vibration  for  other  types  of 
damping 

e 

a 

b 

Use  log  decrement  to 
measure  damping. 
Identify  other  types  of 
damping. 

Derive  the  equation  for 
columb  damping 
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c 

Compute  energy  loss  for 
hysteretic  damping 

4 

Solve  for  forced  vibration  of  Single  Degree 
of  Freedom  Systems 

a 

Derive  the  equation  of 
motion  from  FED  & MAD 

M-30,  M-31,M-32 

b 

i) Use  the  FED  and  MAD 
to  solve  for  the  steady  state 
solution  due  to  harmonic 
excitation 

ii)  Compute  the  total 
response 

hi)  Estimate  damping  ratio 
from  half  power  bandwidth 

M-35,  M-41 

c 

Define  transmissibility  and 
observe  the  effect  of 
damping  and  frequency 
ratios  on  transmissibility. 

M-39,  M-40 

d 

e 

f 

Solve  for  the  response  of  a 
system  due  to  motion  of 
base 

Solve  for  the  response  of  a 
system  subjected  to 
rotating  unbalance 
Design  systems  for  desired 
vibration  isolation. 

M-38 

F-l,F-2,  F-3,  F-4 
F-5,  F-6,  F-7,  F-8,  F-9 

5 

Solve  for  free  vibration  of  a 2DOF  system 

a 

i)  Identify  the  appropriate 
2DOF 

ii)  Construct  FED  and 
MAD  in  terms  of  the 
chosen  2DOF 

F-10,  F-11,F20,  F-21 

b 

Use  the  FED  and  MAD  to 
derive  equations  of  motion 

F-12,  F-13 

c 

Identify  mass,  stiffness, 
and  damping  matrices  from 
the  equations  of  motion. 

F-14,  F-15,  F-22,  F-23,  F- 
28,  F-29 

d 

e 

Solve  for  undamped 
natural  frequencies  and 
mode  shapes. 

Solve  for  modal  properties 
of  a semidefinite  systems 

F-16,  F-17,  F-18,  F-19,  F- 
27 

F-24,F-25,  F-26,  F27,  F- 
30,  F-31,F-32 

6 

Solve  for  forced  vibration  of  2DOF  systems 

a 

Use  the  equation  of  motion  to 
solve  for  steady  state  response 
due  to  harmonic  excitation 
directly  by  impedance  method 

b 

Use  Matlab  to  solve  the 
equations  of  motion  directly- 
by  state  space  method. 
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7 

Solve  for  general  eigenvalue  problem 

a 

b 

Derive  equations  for 
MDOF  systems  & solve 
for  natural  frequencies  and 
mode  shapes  in  closed 
form. 

i)  Check  orthogonalilty  of 
mode  shapes  with  respect 
to  mass  and  stiffness 
matrices 

ii)  Use  the  orthogonality  to 
decouple  equations  of 
motion. 

iii)  Compute  mass 
normalized  mode  shapes. 

F-28,  F-29,  F-30,  F-31,F- 
32 

F-33 

c 

Solve  for  natural 
frequencies  and  mode 
shapes  by  Matlab 

d 

i)  Define  Proportional  and 
non  proportional  damping 

ii) Solve  for  damping  ratios 
for  the  case  of 
proportional  damping 

e 

i)  Decouple  equations  of 
motion  using  separation  of 
variables 

ii)  Solve  for  the  response 
using  mode  shapes  and 
generalized  coordinates 

iii)  Solve  response  of  a 
large  DOF  systems  in 
terms  of  first  few  modes 
and  generalized 
coordinates 

8 

Use  various  vibration  measuring  equipment 

a 

Use  function  generator  to 
generate  a periodic  wave 
and  use  the  analyzer  to 
measure  the  Fourier 
components. 

b 

i)  Measure  natural 
frequency  of  a SDOF 
system  using  impact 
hammer,  accelerometer, 
and  FFT  analyzer 

ii)  Measure  damping  ratio 
using  half  power  points 

F-34,  F-35 
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iii)  Understand  various 
sources  of  error  including 
digital  signal  processing 
issues  and  effect  of 

F-35,  F-36,  F-38 

sensors. 

c 

i)  Measure  natural 
frequencies  and  mode 
shapes  of  a 2DOF  system 
using  impact  hammer, 
accelerometer,  and  FFT 
analyzer. 

F-37 

ii)  Use  imaginary  part  of 
transfer  functions  to  obtain 
modal  parameters. 

iii)  Obtain  damping  ratios 

F-39 

9 

Use  commercial  software  such  as  Matlab 

a 

Reconstruct  a periodic 
wave  from  first  few 
harmonics  and  plot  using 
Matlab 

b 

To  solve  for  complete 
solution  for  response  of  a 
SDOF  using  Matlab 

b 

Solve  for  eigenvalues 
(square  of  natural 
frequencies)  and 
eigenvectors  (mode 
shapes)  of  undamped 
2DOF  and  MDOF  systems 
using  eig  command 

c 

Solve  equation  of  motion 
for  a 2DOF  system  such  as 
automobile  using  state 
space  method. 

10 

Design  a structure  to  prevent  failure  from 
vibration 

a 

i)  Formulate  equation  of 
motion  of  SDOF  system 
and  suggest  appropriate 
stiffness/damping  for 
desired  vibration  isolation 

ii)  Formulate  equation  of 
motion  of  2DOF  systems 
and  suggest  appropriate 
stiffness/damping  for 
desired  vibration  isolation 
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b 

Design  a tuned  absorber 
system  for  vibration 
prevention  at  a resonant 
forcing  frequency  while 
satisfying  design 
constraints  such  as 
maximum  displacement 

11 

Solve  for  forced  vibration  of  a SDOF  system 
due  to  general  forcing  functions. 

a 

Identify  various  sources  of 
periodic  forces  and  use 
Fourier  analysis  to  solve 
for  response  of  SDOF 
system  subjected  to 
periodic  force 

b 

Identify  various  sources  of 
transient  forces  and  solve 
for  response  of  SDOF 
system  subjected  to  the 
transient  forces 

c 

Define  response  spectrums 
and  discuss  their  use  in 
structural  design. 
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B.9.e.5;  TECH  344-Materials  and  Processes  in  the  Plastics  Industry  (Explanation  in  B.9) 


Student  Learning  Objectives  and  Outcomes 

Assessments:  Test  Alignments 
Midterm  (M)  & Final  (F) 

Student  Learning 

Obj  ectives/  Outcomes-Maj  or 

Student  Learning 
Objectives  - minor 

Corresponding  Tests  and 
Test  Items 

A. 

Students  will  Describe  the  Fundamental 

1. 

Students  will  draw  & 

Structure  of  Plastics: 

explain  basic  organic 
molecules. 

a.  Students  will  label  the 

M1,M2,M3,M4 

atoms  and  bonding  types, 
b.  Students  will 

M10,M12,M14 

differentiate  & give 
examples  of  alkanes, 
alkenes,  alkynes,  and 
aromatics. 

F11,F13 

2. 

Students  will  interpret  & 

M16,M17,M18,M19,M20,M21 

draw  polymer  chains, 
a.  Students  will  compare 

M23 

polymerization  reactions, 
b.  Students  will  compare 
& contrast  functional 

M24 

groups  & tacticity. 
c.  Students  will  describe 
chain  topology. 

M26 

3. 

Students  will  compare  & 
contrast  structure  & 

M27,M28,M29,M30,M3 1 

applications  of 
thermoplastics  & 
thermosets, 
a.  Students  will  select 

M33,M34 

commodity  and 
engineered  plastics, 
b.  Students  will 
differentiate  crystalline  & 
amorphous  plastics. 

M37 

4. 

Students  will  name,  draw, 
and  label  elastomers, 
a.  Students  will  explain 
elastomers. 

M38 

b.  Students  will 

M44 

summarize  polyisoprene. 
c.  Students  will  select  & 
qualify  other  elastomers. 

M45 

B. 

Students  will  Predict  Plastics 

1. 

Students  will  describe 

M46 

Properties: 

effects  of  structural 
features  on  plastics 
properties. 
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a.  Students  will  quantify 
& solve  molecular  weight 
distribution. 

M48 

b.  Students  will 
qualitatively  evaluate 
crystallinity  effects. 

M49 

2. 

Students  will  state  force 
and  stress  vs.  strain 

relationship, 
a.  Students  will  name  & 

F50 

illustrate  forces, 
b.  Students  will  subdivide 
stress-strain  curves. 

F51 

c.  Students  will  compare 
stress-strain  curves  for 
different  plastics. 

F52 

3. 

Students  will  distinguish 

F53,F55,F56,F57,F58,F60,F61, 

& explain  mechanical, 
physical,  thermal. 

F62,F64,F54,F59,F63 

environmental,  electrical, 
and  optical  properties, 
a.  Students  will  select 
ASTM  techniques. 

F65 

4. 

Students  will  explain 
interactions  of  modifiers, 
a.  Students  will  classify 
additives,  fillers,  & 
reinforcements. 

F66,F67,F68,F69,F70,F71,F72 

c. 

Students  will  Describe  Plastics  Design 

1. 

Students  will  differentiate 

F73 

and  Finishing  Processing: 

design  methods. 

2. 

Students  will  classify 
ways  of  assembling 
plastics. 

F75,F76,F77,F78.F74 

a.  Students  will  select 
machining  methods. 

F79 

3. 

Students  will  explain 
methods  of  finishing 

F80,F81 

plastics. 

a.  Students  will  give 
examples  of  joining  & 
decorating. 

F82,F83,F84 

4. 

Students  will  compare  & 
contrast  rapid  prototyping 
procedures. 

F85,F86 

D. 

Students  will  Observe  Plastics 

1. 

Students  will  recite 

F87 

Processing  Techniques: 

observed  processing 
methods. 
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E. 

Students  will  Recognize  the 
Environmental  Aspects  of  Plastics: 

1. 

Students  will  recite 
recycling  codes. 

F88,F89,F90,F91,F92 

F93,F94,F95,F97,F96 

2. 

Students  will  explain 
waste  reduction 
techniques. 

a.  Students  will  evaluate 
source  control,  recycling, 
regeneration,  degradation, 
landfills,  & incineration. 

F. 

Students  will  Analyze,  in  Depth, 
Specific  Plastics  Topic: 

1. 

Students  will  construct 
the  history  of  a plastics 
topic,  or 

2. 

Students  will  differentiate 
a plastic,  or 

3. 

Students  will  detail  a 
plastics  processing 
method,  or 

4. 

Students  will  describe,  in 
detail,  a plastic  product. 
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STUDENT  ASSESSMENT  SUMMARY 
Jule  Dee  Scarborough,  Ph.D. 

(See  Portfolio  Sections  B.9.a,b,c,d,  and  B.9.C.1-5;  also,  A. 5 and  A. 7) 


The  goals  of  this  program  component  were  to  diversify  assessment  so  students  had  a wider 
range  of  types  of  assessments  and  more  opportunities  to  provide  evidence  of  learning  - a 
multifaceted  student  assessment  plan  for  the  course.  Another  goal  was  to  improve  the  quality 
of  the  midterm  and  final  exams  by  designing  them  to  incorporate  a wider  range  of  types  of 
items  and  improving  the  problems  to  be  solved,  adding  more  specific  grading  criteria.  Also  it 
was  important  to  measure  the  capability  of  the  assessments  to  engage  students  at  the  upper 
levels  of  Bloom’s  Cognitive  Dimension,  as  tests  and  other  assessments  can  be  designed  at  a 
much  greater  quality  level  when  using  Bloom’s  Cognitive  Dimension  to  determine  if  the 
questions  stimulate  higher  level  thinking  to  reveal  a deeper  understanding  of  the  concepts, 
principles,  knowledge  or  skills.  Finally,  there  was  an  additional  goal  of  adding  three  formal 
performance  assessments,  with  the  corresponding  rubrics  for  scoring  performance  of  each,  to 
the  assessment  plan.  The  above  tasks  were  all  accomplished,  and  once  completed,  the 
professors  analyzed  each  assessment  procedure  against  Bloom’s  Cognitive  Dimension. 

Test  Analysis  and  Development 

During  the  course  analysis  program  component,  the  professors  performed  analyses  of  their 
midterm  and  final  exams  to  use  for  diagnostic  purposes  - for  the  improvement  of  the  tests 
and  instruction.  The  test  analyses  were  used  to  design  and  develop  new  midterms  and  final 
examinations  for  the  2006  course  to  be  taught  during  the  experimental  research  semester. 
Therefore,  regarding  the  timeline,  the  test  analyses  were  performed  after  the  course,  teaching, 
and  learning  analyses  were  completed.  (See  the  GAPS  Analysis  in  Portfolio  Section  B.5  for 
those  results.) 

After  identifying  the  strengths  and  weaknesses  of  the  tests  used  in  the  2005  course,  the 
professors  then  developed  new  Student  Learning  Outcomes  for  the  2006  course,  as  described 
in  the  previous  Outcomes  section.  In  the  “Reversed  and  Intentional  Instructional  Design” 
process,  the  Outcomes  define  the  “what”  is  to  be  learned  and  then  what  type  of  assessments 
are  to  be  used  for  measuring  student  learning  is  determined.  This  answers  the  question  “what 
is  acceptable  evidence  of  learning?”  The  professors  designed  and  developed  new  objective 
midterm  and  final  examinations;  some  professors  included  subjective  items  as  well.  (Refer  to 
the  section  following  this  summary  to  read  the  analysis  results  for  the  2005  tests  as  compared 
to  the  new  2006  tests.  Portfolio  Section  B.8.2.) 

Briefly,  many  of  the  professors  included  mathematical  problems  on  their  2005  tests,  some 
had  objective  items,  and  a few  had  no  objective  items.  The  range  of  types  of  tests  varied 
across  professors.  The  strength  of  some  tests  might  be  the  problems,  except  that  there  were 
no  grading  criteria  or  rubrics  with  standards  or  criteria  of  performance.  Therefore,  the 
grading  of  problems  was  not  perceived  as  consistent  nor  could  it  be  determined  exactly  what 
professors  were  seeking  beyond  the  “right”  answer.  Some  professors  did  give  points  along 
the  way  for  “right”  partial  elements  of  solutions;  others  did  not.  The  grading  was  far  too 
unspecified  and  far  too  subjective  to  really  mean  much.  Also  the  point  distribution  or  grade 
distribution  was  not  formulated  based  upon  objective  criteria.  Finally,  for  most  of  the 
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courses,  the  tests  seemed  to  be  the  primary  student  assessments,  with  very  few  (possibly 
homework)  other  opportunities  to  further  assess  student  learning.  Several  did  include  design 
projeets,  but  these  also  were  missing  formal  grading  formula,  criteria,  or  rubries.  Once  again 
the  grading  was  perceived  as  far  too  subjective,  and  the  potential  ineonsisteneies  and  laek  of 
formal  and  predetermined  objectivity  left  grading  ambiguous.  Professors  laeked  a 
multifaceted  assessment  plan  for  their  courses  where  students  were  assessed  for  learning 
using  a variety  of  types  of  assessments.  They  also  provided  few  opportunities  for  students  to 
demonstrate  evidence  of  learning,  meaning  that,  there  were  not  very  many  assessments. 
Students  are  at  a real  disadvantage  where  there  are  very  few  assessments. 

Once  the  2005  test  analyses  were  completed  and  we  determined  that  there  were  no 
performance  assessments  in  any  of  the  eourses,  the  professors  designed  and  developed  new 
midterm  and  final  exams  for  the  2006  course.  Using  the  results  of  those  analyses 
diagnostically  and  the  new  2006  student  learning  outcomes,  the  professors  developed  a Table 
of  Specifieations  to  guide  their  creation  of  a new  midterm  and  final  exam  for  the  2006 
eourse.  They  eaeh  developed  an  objeetive  test  item  bank  of  multiple  questions  for  eaeh 
student  learning  outcome.  Once  the  objective  test  items  were  developed,  they  chose  items  for 
each  exam,  midterm  and  final,  and  assembled  the  tests.  If  they  preferred  to  include  problems 
to  solve,  those  were  added  as  well.  The  program  leaders  provided  feedback  throughout  the 
entire  analysis  and  development  process.  To  further  ensure  that  the  test  items  and  assembled 
tests  actually  measured  knowledge  or  skills  inherent  in  an  outcome,  the  professors  mapped 
the  outcomes  to  the  eorresponding  tests  and  speeific  items.  This  helped  them  realize  where 
they  needed  more  items  or  a different  type  of  item  and,  especially,  where  there  were  gaps  in 
the  measurement  of  critieal  outcomes.  (See  worksheet  below  and  other  examples  in 
B.9.2.e.l-5.)  Although  the  tests  were  not  perfect,  they  were  greatly  improved.  (See  Portfolio 
Section  B.9.a  for  the  comparison  for  differences  between  the  2005  and  2006  tests;  see 
Program  Deseription,  Portfolio  Seetion  A. 7,  for  further  information  on  the  Test  Analysis  and 
Development  program  eomponents.) 

An  example  of  the  outcome  to  test  and  test  item  analysis  is  copied  here. 
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Table  B.9.1:  lENG  475  - Decision  Analysis  - Regina  Rahn 


Student  Learning  Objectives  and  Outcomes 

Assessments: 

Test  Alignments 
Midterm  & Final 

Student  Learning 

Obj  ectives/  Outcomes-Maj  or 

Student  learning  objectives  - minor 

Corresponding 
Tests  and 
Test  Items 

I 

To  learn  to  use  a specific  set  of  analytical  tools 
for  technical  decision  making  under  uncertainty. 

A 

Students  will  be  able  to  construct/create  a 
decision  tree  to  aid  in  determining  the  best 
course  of  action  for  a given  set  of 
circumstances 

1 . To  define  the  states  of  nature  of  the 
system,  process,  or  situation 

2.  To  develop  the  branch  structure  of  the 
tree 

a.  To  identify  decision  nodes; 
what  are  the  items  the  decision 
maker  chooses 

b.  To  identify  the  chance  nodes; 
the  events  that  occur  by  chance 
with  a given  probability 

c.  To  draw  the  arcs,  which  define 
the  sequences  and  relationships 
between  nodes 

3.  To  identify  the  outcomes 

a.  To  define  the  choices  for  a 
decision  node 

b.  To  define  the  possible 
outcomes  of  a chance  node, 
which  are  a set  of  mutually 
exclusive  outcomes 

c.  To  define  the  “consequence,” 
or  the  final  outcome  of  a 
branch 

4.  To  solve  for  the  expected  value  of  the 
decision  tree  (EV,  EMV) 

a.  To  construct  the  joint, 
conditional,  and  marginal 
probabilities 

b.  To  calculate  all  branch 
probabilities  of  the  tree 

i.  To  apply  B aye’s  Theorem 

ii. To  implement  the  inverse 
tree  structure  technique 

5.  To  find  and  compare  the  expected  value 

of  both  sample  and  perfect 
information  (EVPI,  EVSI) 

a.  To  construct  the  decision  trees 
to  calculate  EVPI  and  EVSI 

b.  To  evaluate  the  relevance  and 
importance  of  the  values 
obtained  for  EVPI  and  EVSI  to 
the  decision  process 

HW  #1  (A) 

HW  #2  (A) 

HW  #3  (B,C) 

HW  #4  (D) 

HW  #5  (E) 

Performance  Task 
#1  (A) 

Midterm  #1-18 
(A) 

Midterm  #19- 
21(B) 

Midterm  # 22-30 
(C) 

Pinal#  19-22  (A) 

Pinal  #6-10,  23-25 
(C) 

Pinal#  1-5  (D) 
Pinal#  11-18  (E) 
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Performance  Assessment  (See  B.9.b  and  c;  also,  A. 5 and  A.7) 

The  second  major  program  component  related  to  student  assessment  was  Performance 
Assessment.  Each  professor  developed  three  performance  tasks  and  corresponding  rubrics. 
One  performance  task/rubric  corresponded  with  the  midterm  examination  and  a second 
corresponded  with  the  final  exam.  The  third  performance  task/rubric  use  and  focus  was 
determined  by  the  professors.  Why  design  the  two  performance  tasks  to  correspond  to  the 
midterm  and  final  examinations  - because  the  underlying  premise  that  we  chose  to  accept 
was  that  “most”  or  “typical”  tests,  at  best,  provided  evidence  of  what  students  may  “know 
about”  or  “know”  at  the  lower  levels  of  Bloom’s  Cognitive  Dimension  (memorize  or  limited 
comprehension).  We  do  acknowledge  that  good  tests  may  achieve  these  Bloom’s  levels,  but 
tests  do  not  “usually”  provide  the  opportunity  for  students  to  apply,  analyze,  evaluate, 
synthesize  or  create.  Furthermore,  if  well  designed  and  developed  problems  are  used,  either 
for  students  to  solve  with  established  performance  criteria  forjudging  performance  or  to  have 
to  figure  out  before  responding  to  a selection  of  provided  responses,  then  these  well  designed 
and  developed  tests  can  accomplish  the  goal  of  providing  high  quality  evidence  of  learning. 
But  developing  such  tests  really  requires  knowledge  and  skill  in  test  analysis  and 
development,  as  well  as  a great  deal  of  time.  When  considering  that  most  engineers  and 
technology  professors,  as  well  as  most  of  the  university  population  of  professors,  have  little 
to  no  background  in  educational  or  learning  theory  or  in  student  assessment,  then  the  reality 
is  that  most  tests  may  not  measure  anything  of  significance  very  well  or  provide  any  worthy 
evidence  of  learning.  Therefore,  if  one  accepts  that  well  designed  and  developed  performance 
tasks  (authentic  and  real  world),  with  high  quality  and  well  defined  performance  criteria 
through  the  use  of  rubrics,  can  provide  a greater  opportunity  for  students  to  provide  evidence 
of  what  they  can  “do”  with  the  knowledge  and  skills,  then  performance  assessment  extends 
the  evidence  of  learning  possibilities.  However,  performance  tasks  and  objective  tests,  even 
with  good  problems,  can  measure  some  similar  things,  but  also  very  different  knowledge  and 
skills.  Thus,  we  wanted  the  performance  tasks  to  relate,  but  also  to  extend  the  evidence 
learning  to  include  “doing”  or  “performing.”  We  had  hoped  to  consider  the  difference  and 
similarities  between  the  two  types  of  measures,  a good  study  in  itself  if  formally  executed. 
But  we  realized  that  the  professors  did  not  have  background  enough  in  assessment  or  the 
theory  needed  to  design  such  research  as  this  point  in  their  experience.  However,  this  was  our 
way  of  introducing  performance  tasks/rubrics  while  also  asking  the  professors  to  improve 
their  tests.  Some  of  the  professors  did  not  have  objective  test  items  at  all  on  their  tests;  there 
were  just  subjective  problems  to  solve  without  any  established  performance  criteria.  We 
wanted  them  to  understand  that  if  they  were  going  to  use  solving  a problem  as  a high  quality 
performance,  then  they  had  to  go  the  next  step  and  develop  a rubric  that  revealed  the 
performance  criteria  for  solving  the  problem,  making  it  clear  to  both  the  student  and  the 
professor  what  the  standards  and  criteria  of  performance  could  be.  This  also  helped  them  to 
understand  that  they  needed  to  grade  more  than  just  the  end  result  or  answer,  that  problem 
solving  was  a process  that  resulted  in  an  answer. 

For  this  first  venture  into  performance  assessment,  we  required  three  “complex”  performance 
tasks,  where  each  performance  task  was  actually  a cluster  of  several  performances,  was 
authentic,  and  reflected  real  world  tasks,  and  the  rubrics  revealed  levels  of  possible  standards 
for  performance  and  the  criteria  for  each  standard.  Each  professor  developed  the  three 
performance  tasks  with  corresponding  rubrics. 
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Both  professors  and  students  reported  liking  performance  assessment.  The  first  ones  were 
somewhat  simple,  but  that  will  change  as  the  professors  continue  to  modify  them  and  become 
more  skilled  at  developing  the  tasks  and  rubrics.  All  professors  reported  that  they  will 
continue  to  use  performance  tasks  and  rubrics.  This  type  of  assessment  greatly  improved 
their  overall  course-student  learning  assessment  plans,  further  diversifying  the  types  of 
assessment  for  the  course.  Performance  tasks  also  added  more  opportunities  to  provide 
evidence  of  learning.  (See  the  Assessment  Plan  Maps  included  below,  example  and 
professors’.)  Finally,  each  professor  studied  his/her  overall  tests,  performance  tasks,  and  any 
other  types  of  assessment  they  included  (e.g.,  design  projects,  collaborative  projects,  etc.)  and 
analyzed  them  for  achieving  the  higher  levels  on  Bloom’s  Cognitive  Dimension.  There  was  a 
greatly  improved  range  of  Bloom’s  upper  levels.  For  most  of  the  professors,  the  tests 
achieved  well  to  the  application  level,  with  three  midterms  and  finals  achieving  the  analysis 
and  synthesis  levels.  Clearly,  progressing  to  the  inclusion  of  formal  performance  tasks  moved 
the  assessment  plans  to  the  higher  levels  of  Bloom’s  Cognitive  Dimension. 


The  Chart  has  been  copied  into  this  document  helow  for  reader  convenience. 
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Table  B.9.2:  Bloom’s  Learning  Taxonomy  -Assessment  Analysis  Chart  for  2006 
CITL  Professors  (numerical  reported  in  %) 


Assessment 

Knowledge 

Comprehension 

Application 

Analyze 

Synthesize 

Evaluate 

Remember 

Understand 

Apply 

Analyze 

Evaluate 

Create 

Midterm  (1) 

15% 

30% 

7.5% 

7.5% 

35% 

None 

(2) 

1 

15 

7 

69 

7 

None 

(3) 

5 

35 

45 

10 

5 

None 

(4) 

10 

35 

50 

5 

None 

None 

(5) 

None 

10 

50 

40 

None 

None 

(6) 

8 

38 

54 

None 

None 

None 

(7) 

30 

70 

none 

none 

none 

none 

Final  (1) 

none 

24% 

none 

12% 

24% 

46% 

(2) 

2 

14 

8 

76 

None 

None 

(3) 

NR 

NR 

NR 

NR 

NR 

NR 

(4) 

5 

30 

55 

10 

None 

None 

(5) 

None 

10 

50 

30 

10 

None 

(6) 

4 

35 

61 

None 

None 

None 

(7) 

30 

70 

none 

none 

none 

none 

Performance  (1) 

10% 

10% 

10% 

20% 

20% 

30% 

Taskl 

(2) 

None 

None 

10 

30 

30 

30 

(3) 

None 

None 

None 

50 

50 

None 

(4) 

None 

None 

None 

None 

100 

None 

(5) 

None 

None 

20 

20 

50 

10 

(6) 

None 

None 

None 

None 

50 

50 

(7) 

None 

none 

none 

none 

50 

50 
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Assessment 

Knowledge 

Comprehension 

Application 

Analyze 

Synthesize 

Evaluate 

Remember 

Understand 

Apply 

Analyze 

Evaluate 

Create 

Homework 

(1) 

none 

40% 

5% 

30% 

10% 

15% 

(2) 

10 

20 

30 

40 

none 

none 

(5) 

None 

30 

50 

20 

None 

None 

(6) 

None 

40 

30 

30 

None 

None 

(7) 

X 

X 

none 

none 

none 

none 

Lab 

Experiments 

(2) 

none 

20 

40 

30 

10 

none 

Miscellaneous: 

Individual 

(4) 

none 

none 

none 

33 

33 

33 

Assessments 

(6) 

Group 

Assess- 

(4) 

none 

none 

none 

33 

33 

33 

Ments 

(6) 

Group 

Discussion 

(5) 

None 

None 

20 

50 

30 

None 

Round  Table 
Discussion 

(5) 

None 

None 

20 

50 

30 

None 

Oral 

Presentation 

(5) 

none 

none 

none 

10 

40 

50 

During  the  2006  experimental  research  semester,  the  professors  used  their  new  assessments  to 
measure  and  provide  evidence  of  student  learning.  After  the  midterm  and  final  exams,  they 
performed  an  analysis,  as  they  did  initially  with  the  2005  tests.  This  process  moved  them  into 
further  use  of  test  analysis  for  diagnostic  purposes  to  improve  tests,  instruction,  and  student 
learning.  They  also  performed  their  first  diagnostic  review  of  the  performance  tasks, 
qualitatively,  to  determine  strengths  and  needed  improvements.  Both  of  these  processes  will 
assist  them  in  improving  the  assessments  for  the  next  course  offering  and  further  instill  the  best 
practices  of  test  analysis  for  diagnostic  purposes,  the  use  of  performance  tasks/rubrics,  and  the 
analysis  of  the  performance  tasks/rubrics  (also  for  diagnostic  purposes)  to  improve  assessment 
and  instruction,  and  ultimately  student  learning. 
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This  was  a successful  program  component,  and  there  was  significant  gain  in  knowledge  and 
skills  by  all  professors.  See  following  sections: 

• Bloom’s  Analysis  of  Assessment  Plans  (copied  into  this  document  above) 

• Final  Report:  Discussion  of  Professors’  2005  and  2006  Test  Analyses  and  Tests 
(See  B.9.a) 

• Professor  Diagnostic  Analyses  of  their  2006  midterm  and  final  exams. 

(See  B.9.d) 

• Professors’  Multifaceted  Assessment  Plan  Graphics  (samples  are  copied  into  this 
document  below)  The  Maps  are  based  upon  Kuhs  et  al.  (2001)  and  Scarborough 
(2006)  modifications  for  the  Faculty  Development  Program  (Tables  B.9.3).  The 
Program  sample  is  below,  followed  by  each  of  the  Faculty  participants  (Tables 
B.9.4-9). 

Note:  We  were  going  to  explore  studying  the  correlations  between  the  Objective  Tests  and 
the  Performance  Assessments.  We  did  compute  the  correlations,  but  the  research 
considerations  were  too  complex  to  consider.  However  refer  to  B.9.c  for  the  discussion. 


Table  B.9.3 
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CEET  Professors  2006  Multifaceted  Assessment  Plans 

Maps  based  on  Scarborough,  2006  and  Kuhs  et  al,  2001 


Multifaceted  Assessment  System 

Azad,  Abul,  2007,  Technology  277 


Ibrahim  Abdel-Motaleb 
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Multifaceted  Assessment  Plan  - Map 
IENG370,  Operations  Research  - R.  Moraga,  2006 
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I 


I 


A^nswlr  I ( Long  Problems  ] 


Flow  Charting  I 


Mapping 


Observation 


I 


[ Group  Discussion  ] ^ Rating  Scale  ] 


[ Research  ] [ Design  ] [jeam  Performance] 


»jopen-l 


Selected  Response  Items 


|Multiple  Choicej 

c 


Matching 


Written  H — Performance 

' Assessment 

— 


Multifaceted  Assessment  System 


[ Papers  (2)  ] 


( Or^\  ) 

— 


Demonstration/Product  I 


1 ( * ^ f Formal  Written  Report  1 

[ Short  Summary  J I Short  Presentation  I ^ ^ 

i 

I Formal  PresentatioTi^ 


lENG  475  - Fall  2006 
Gina  Rahn 


Multifaceted  Assessment  Plan,  MEE  321,  Abhijit  Gupta,  2006 
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Faculty  Participants 

Abul  Azad  ^ ^ / 1 • I 

Brian  Coller  U juove  leachiriQ  at 

Abhijit  Gupta 
Reinaldo  Moraga 
Ibrahim  Abdel-Motaleb 
Regina  Rahn 
Robert  Tatara 


» . 

Volume  III:  Program  Worksheets;  Forms;  and 


The  Scholarship  of  Teaching 
The  CEET  Initiative  on  Teaching  and  Learning 


Jule  Dee  Scarborough,  Initiative  Director 

' »■ 

'V 

' College  of  Engineering  and  Engineering  Technoiogy 


Promod  Vohra,  Dean 


Program  Associate:  Jerry  Gilmer,  Director,  NIL!  Testing  Services 
Editor:  Gail  Jacky,  Director,  NIL!  Writin'^»4^enter 


■'S- 


Discipline  Course  Outline 


Course  Disciplinary 

Science(s) 

Mathematics 

Communication  Foundation/ 

Content 

Foundation  Required 

(Id.  Physics,  Chemistry,  Biology,  etc.) 

Foundation  Required 

Skills  Required 

I.  Unit 

A. 

A.  Physics,  Chemistry,  Bio? 

A. 

1. 

A. 

1. 

1. 

2. 

1. 

2. 

2. 

B. 

1. 

2. 


C. 

1. 

2. 

II.  Unit 

III.  Unit 

IV.  Unit,  etc. 
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Table  C.1.2  ; Content  Schedule  and  Styles,  Models,  Bloom's  Analysis 


Week 

Content  Topic: 

Factual,  Conceptual, 
Procedural,  Meta-cognitive 

Content 

Source 

Text,  etc. 

Teaching 

style 

a-k;fpfd 

Learning 
style  CE, 
AE,  AC,  RO 

Teaching 

Model 

1-24  name 

Dale's  Cone 
Active  or  Passive 

Bloom's  Traditional:  Evaluation, 

Synthesis,  Analysis,  Application, 
Comprehension , Knowledge 

Bloom's  Revised 

Create,  Evaluate,  Analyze,  Apply, 
Understand,  Remember 

Critical  Thinking 

Centered? 

Teacher,  Knowledge 
Assessment,  Learner 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

13 

14 

15 

16 

Final  Exam  or  Project 

Note:  There  is  a practice  bv  some  professors  to  give  the  final  exam  before  or  during  the  last  week  of  CLASS,  rather  than  the  FINAL  EXAM  WEEK.  This  essentially  means  that  vou  are  "cheating"  the  students  out  of  one  dav  of  content,  learning,  etc.  We  should  have  15  weeks  of  learning, 
including  exam,  quizzes,  or  project  days,  but  to  make  the  final  week  of  class  the  FINAL  exam  week  is  unethical  by  NIU  standards,  regardless  of  who  does  it.  What  is  your  practice? 
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12 

13 

14 

15 

16 

Final  Exam  or  Project 

Note:  There  is  a practice  bv  some  professors  to  sive  the  final  exam  before  or  during  the  last  week  of  CLASS,  rather  than  the  FINAL  EXAM  WEEK.  This  essentially  means  that  you  are  "cheating"  the  students  out  of  one  day  of  content,  learning,  etc.  We  should 
haye  15  weeks  of  learning,  including  exam,  quizzes,  or  project  days,  but  to  make  the  final  week  of  class  the  FINAL  exam  week  is  unethical  by  NIU  standards,  regardless  of  who  does  it.  What  is  your  practice? 
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Table  C.1.4:  Instructional  Design  Gaps  Analysis  Table 


ABET/ 
NAIT 
Standard 
a-k  Eng 
A-Q  Tech 

NIU  General  Ed 
Goals  (embedded) 
a-I,  ii,  iii,  iv 
b-I,  ii,  iii,  iv 
c and  d 

Student  Learning 
Objectives 
listed  on  syllabus 

Bloom/Dale 

Evaluation/Active 

Synthesis/Active 

Analysis/Active 

Application/Active 

Comprehension/P 

Knowledge/Passive 

Knowledge 

Sources 

Professor,  Text, 
Cases,  Speaker, 
References,  etc. 

Student 
Assessments 
listed  on 
syllabus 

Bloom/Dale 

Evaluation/Active 

Synthesis/Active 

Analysis/Active 

Application/Active 

Comprehension/P 

Knowledge/Passive 

Test 

Items 

or 

Projects/ 

Rubrics 

Bloom/Dale 

Evaluation 

Synthesis 

Analysis 

Application 

Comprehension 

Knowledge 

Performance 
IF  any; 
if  none, 
leave 
blank 

Bloom/Dale 

Evaluation/Active 

Synthesis/Active 

Analysis/Active 

Application/Active 

Comprehension/P 

Knowledge/Passive 
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PROFESSORS  — READ  OUT  LOUD  CAREFULLY  to  students  before  handing  out  questionnaires. 


To:  Participating  students 
Fr:  Dean 

Re:  New  initiative  to  study  the  quality  of  instruction,  course  content,  and  the  learning  environment 
across  the  college 


The  following  questionnaire  is  being  administered  to  selected  classes  across  the  college  and  its  four 
departments.  As  students  in  those  classes,  you  are  being  asked  to  participate  in  providing  baseline 
information  about  the  quality  of  instruction,  course  content,  and  the  learning  environment  in  the  college. 

It  is  important  to  take  note  of  the  following: 

(a)  The  first  part  of  the  questionnaire  relates  only  to  this  course. 

(b)  The  second  part  of  the  questionnaire  relates  to  your  experience  across  all  the  courses  you  have  taken 
in  your  major  department. 

(c)  This  questionnaire  does  not  seek  information  about  your  experience  in  any  courses  outside  the  major 
department,  e.g.  general  education  or  courses  transferred  to  NIU. 

Your  responses  to  these  questions  will  be  used  as  baseline  information  to  study  how  to  strengthen  the 
quality  of  education  across  the  college. 

We  are  hoping  that  you  will  complete  this  questionnaire  thoughtfully,  seriously,  and  genuinely,  with  the 
understanding  that  it  is  important  and  will  assist  us  in  structuring  a college  initiative  to  study  and 
strengthen  the  quality  of  instruction,  course  content,  and  the  learning  environment  across  the  college. 

In  testing  the  questionnaire  with  students,  it  took  about  20  minutes,  therefore,  we  are  allowing  30  minutes 
of  class  time  to  complete  the  questionnaire  in  class. 

The  questionnaire  is  somewhat  long,  but  not  as  long  as  it  may  seem,  because  the  questions  have  been 
written  in  a way  that  hopefully  describe  thoroughly  what  we  are  seeking  information  about.  Also,  the 
print  is  regular  sized  and  we  have  spaced  and  printed  the  document  for  easier  reading. 

Please  attend  to  each  item  carefully  and  respond  to  the  best  of  your  ability.  We  need  your  input.  It  is 
important  that  you  respond  honestly,  genuinely,  and  with  sincerity  as  the  results  of  the  survey  will  greatly 
impact  the  Dean's  new  initiative  on  the  quality  of  education  for  students  in  the  college. 

Thank  you  for  investing  your  time  and  serious  effort  to  help  us  begin  this  very  important  initiative. 
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Course  Questionnaire  on  Teaching  and  Learning 

Jule  Dee  Scarborough  (2006) 

After  completing  the  student  and  course  information  on  the  front  side  of  the  scan  form,  respond  to  the  following  questions 
on  the  back  side  of  the  form  beginning  with  item  101. 

Questions  101-124  focus  on  the  course  you  are  now  ending.  Please  respond  to  101-124  based  upon 
your  experience  in  this  course  only. 

101.  The  course  syllabus  identified  specific  learning  objectives. 

a.  Yes,  and  I understood  them 

b.  Yes,  but  I didn't  understand  them 

c.  I don't  know 

d.  No,  there  were  no  learning  objectives 

102.  The  learning  objectives  for  this  course  were  chosen  or  required  by:  (Select  all  that  apply.) 

a.  Future  employers 

b.  Department  head 

c.  Professor's  interests 

d.  Accreditation  agency 

e.  NIU  General  Education  Goals 

f.  I don't  know 

103.  The  course  syllabus  specified  (Select  all  that  apply) 

a.  course  or  student  learning  objectives 

b.  course  description 

c.  clearly  defined  course  content 

d.  clearly  defined  assignments,  labs,  papers,  projects,  tests,  or  other  important  assignments  or  activities 

e.  the  course  schedule  or  timeline  identifying  meeting  dates,  assignment  due  dates,  and  the  semester's 

schedule 

f.  additional  explanations  of  course  requirements  which  established  the  criteria  for  each  assignment 

g.  references  other  than  the  text  (e.g.,  books,  websites,  articles,  other  sources  related  to  course  content) 

h.  contact  information  for  professor,  instructor,  and/or  graduate  teaching  or  lab  assistants 

104.  The  professor  (and  any  assistants)  (Select  all  that  apply) 

a.  focused  content  and  learning  activities  on  the  course  or  student  learning  objectives  throughout  the 

semester 

b.  provided  learning  that  seemed  to  align  with  the  course  description 

c.  taught  the  course  content  specified  in  the  syllabus 

d.  followed  the  assignments,  labs,  papers,  projects,  tests,  or  other  important  assignments  or  activities 

outlined  and  defined  in  the  syllabus 

e.  followed  the  course  schedule  or  timeline  specified  in  the  syllabus  (e.g.,  meeting  dates,  assignment  due 

dates,  and  the  semester's  schedule) 

f.  graded  assignments  according  to  the  written  explanations  for  course  requirements  establishing  the 

criteria  for  each  assignment 

g.  were  available  using  the  contact  information  for  professor,  instructor,  and/or  graduate  assistants 

h.  deviated  from  the  syllabus  by  adding  appropriate  content  to  expand,  deepen  understanding,  or 

resolve  questions  resulting  in  adding  value  to  the  course;  any  additional  assignments  were 
appropriate  having  reasonable  timelines 

i.  deviated  from  the  syllabus  inappropriately,  where  additions  to  the  information  provided  on  the 

syllabus,  or  new  assignments  added  were  irrelevant  or  distracting  and  added  little  or  no  value  to 
the  course  or  learning;  new  assignments  were  untimely  and  caused  unnecessary  stress  for  students 

j.  The  course  was  well  organized,  structured,  and  executed. 
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105.  Which  of  the  following  methods  were  used  hy  the  professor  to  measure  learning?  (Select  all  that  apply) 

a.  final  exam  - traditional  test 

b.  midterm  exam  - traditional  test 

c.  quizzes  and/or  short  tests  periodically-traditional  test(s)  (e.g.,  multiple-choice  or  true/false) 

d.  quizzes  and/or  short  tests  periodically  - short  answer  and/or  essay 

e.  research  or  learning  paper  (s),  usually  requiring  literature  search  or  field  research  and  formal  write-up 

f.  case  study(ies)  in  industry,  usually  requiring  a report  or  short  paper  write-up 

g.  hands-on  technical  project(s) 

h.  hands-on  non-technical  project(s) 

i.  other  types  of  performances,  "doing"  something 

j.  course  portfolio,  full  documentation  of  all  work  and  progress  in  the  course 

k.  Other;  write  a description  here: 

106.  Select  ALL  the  descriptions  helow  that  identify  the  methods  being  used  in  this  course  to  measure 

student  learning: 

a.  Learning  was  measured  on  my  ability  to  memorize  terminology,  symbols,  facts,  information,  theory, 
principles,  concepts,  information,  definitions,  descriptions 

b.  Learning  was  measured  on  my  ability  to  make  comparisons  to  determine  similar  and  dissimilar 
examples,  understanding  relationships  and  connections  between  and  among  facts,  concepts,  theories, 
principles,  translates  knowledge  into  a new  context,  interpret  facts,  predict  consequences,  order,  group 
information,  contrast,  distinguish,  estimate,  differentiate,  discuss,  or  extend  knowledge 

c.  Learning  was  measured  on  my  ability  to  use  information,  methods,  concepts,  theories  in  new  situations; 
problem  solving  - this  requires  choosing  and  applying  knowledge  (e.g.,  the  best  formula,  concept, 
principle,  theory  to  solve  problems)  and  using  inductive  reasoning  to  determine  tbe  best  methods, 
techniques,  tools,  and  strategies  to  apply  towards  a best  solution;  this  method  of  measurement  can  range 
from  a test  item  with  a complex  problem  to  be  solved  or  a hands-on  technical  problem,  requiring  tbe 
design  and  building  of  something  mechanical.  The  key  to  this  method  is  that  it  requires  application  of 
knowledge  - "doing"  (demonstrate,  calculate,  illustrate,  show,  solve,  examine,  modify,  relate,  change, 
experiment,  discover) 

d.  Learning  was  measured  on  my  ability  to  recognize  patterns  in  information,  problems,  and  situations; 
tbe  ability  to  organize  parts,  identify  or  discover  "bidden"  meanings,  identify  components;  this  requires 
one  to  analyze,  separate  thoughts,  processes,  problems,  order,  explain,  connect,  classify,  and  divide, 
compare,  select,  explain,  and  make  inferences  (indirect  meanings);  this  requires  deductive  reasoning 
where  one  begins  with  facts  and  information  and  makes  choices  to  gradually  discover  the  bigger  picture 

e.  Learning  measured  my  ability  to  hypothesize,  design,  support  argument,  schematize,  write,  report, 

justify,  choose,  evaluate,  estimate,  judge,  criticize,  defend,  and  use  old  ideas  to  create  new  ones, 
extending  the  old  idea  into  a new  one  for  extended  applications  and  to  make  choices  based  upon 
reasoned  argument,  verify  value  of  evidence,  recognize  when  subjectivity  is  being  used  rather  than 
objectivity  (more  scientific),  make  sound  generalizations  from  given  facts,  relate  and  use  knowledge 
across  different  contexts,  predict  and  draw  conclusions,  combine,  integrate,  modify,  rearrange, 
substitute  knowledge,  plan,  formulate,  compare  and  discriminate  between,  summarize,  and  make 
conclusions 

f.  Learning  measured  my  ability  to  design,  discover,  invent,  develop,  create,  research;  transform 
knowledge  into  a product,  process,  technique,  model,  method,  strategy,  etc. 
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107.  Select  the  response  that  best  describes  the  relationship  between  the  traditional  tests  you  have  taken  to 
date  in  this  course  (e.g.  multiple-choice,  true/false  items,  etc.)  and  the  course  content. 

a.  the  content  of  the  test(s)  was  related  to  the  content  specified  in  the  syllahus  and  only  to  content  specified  in  the 
syllahus. 


h.  the  content  of  the  test(s)  was  related  to  the  content  specified  in  the  syllahus  and  other  content  proyided  hy  the 
professor  or  assistants. 

c.  the  content  of  the  test(s)  did  not  relate  to  the  content  specified  in  the  syllahus  but  did  relate  the  other  content 

proyided  hy  the  professor  or  assistants. 

d.  the  content  of  the  test(s)  related  to  neither  (1)  the  content  specified  in  the  syllahus  nor  (2)  the  other  content 

proyided  hy  the  professor  or  assistants. 


Items  108-111  relate  to  the  measurement  of  student  learning  through  performance(s)  rather  than  traditional 
tests.  ***  Consider  the  definitions  below  when  responding  to  items  108-111. 

***  Definitions: 

***Performance  Task  (or  assessment)  - any  authentic  or  real-world  task  designed  to  measure  student  learning. 
Such  a task  can  be  used  to  determine  what  students  can  "do"  with  knowledge.  Unlike  some  traditional  tests, 
performance  tasks  require  students  to  move  to  another  level  of  providing  evidence  of  learning  - that  of  applying  or 
using  knowledge  by  performing  authentic  tasks,  such  as  designing  a part  or  product,  or  designing  and  then 
producing  the  part  or  product.  Writing  a paper  would  provide  evidence  of  research  skills  and  communication  skills, 
for  example. 

(108)  Performance  tasks  were  used  to  measure  student  learning  in  this  course.  (*see  definition  above) 

a.  Yes  (according  to  the  definition  above) 
h.  No  (according  to  the  definition  above) 

***Rubric  - any  type  of  information  sheet  or  form,  check  off  sheet  that  establishes  the  levels  of  performance 
criteria  for  performance  tasks;  these  criteria  establish  standards  for  performance  and  the  criteria  for  each 
standard;  they  are  used  to  provide  students  information  about  what  is  required  to  achieve  a particular  number  of 
points  or  grade.  See  attached  example  at  end  of  questionnaire  following  this  page;  then  continue  to  complete  the 
questionnaire.. 

(109)  Rubrics  were  used  for  scoring  or  grading  the  performances  in  this  course. 

a.  Yes  (according  to  the  definition  below) 

b.  No  (according  to  the  definition  below) 

(110)  Below  are  examples  of  some  performance  tasks.  Identify  any  that  are  similar  to  performances  that 
you  had  to  accomplish  during  this  course.  Select  all  that  apply: 

a.  Writing  a paper 

b.  Working  problems,  showing  the  entire  equation  worked  out  manually,  through  each  step  of  the  equation 

c.  Designing  a product  part,  entire  machine,  other  major  design  project 

d.  Designing  an  industrial  production  system 

e.  Designing  electrical  circuitry  or  full  electrical/electronic  system 

f.  Designing  and  producing  a part  using  manufacturing  processes  (e.g.,  actually  producing  product  using 

manufacturing  production  equipment  in  a lab  or  on-site  in  industry) 
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(Ill)  Select  all  examples  of  performance  tasks  below  (similar)  where  a rubric  or  performance  criteria  form 
was  used  to  score  or  grade  the  performance(s)  during  this  course. 

a.  Writing  a paper 

b.  Working  problems,  showing  the  entire  equation  worked  out  manually,  through  each  step  of  the  equation 

c.  Designing  a product  part,  entire  machine,  other  major  design  project 

d.  Designing  an  industrial  production  system 

e.  Designing  electrical  circuitry  or  full  electrical/electronic  system 

f.  Designing  and  producing  a part  using  manufacturing  processes,  e.g.  actually  producing  product  using 

manufacturing  production  equipment  in  a lab  or  on-site  in  industry 

RUBRIC  COPIES  for  STUDENT  VIEWING  HERE 

112.  The  following  items  related  to  levels  of  learning  and  how  learning  takes  place. 

(Select  ALL  that  apply) 

a.  the  learning  of  basic  knowledge  requiring  me  to  list,  name,  identify,  show,  define,  recognize,  recall, 
state,  visualize,  state  facts,  concepts,  theories,  principles,  information? 

b.  the  comprehension  or  greater  understanding  of  knowledge  through  activities  that  required  me  to 
summarize,  explain,  interpret,  describe,  compare,  paraphrase,  differentiate,  demonstrate,  classify,  or 
contrast  facts,  information,  concepts,  theories,  and  principles? 

c.  the  application  or  opportunity  to  “do”  or  “perform”  using  knowledge,  requiring  me  to  solve  problems, 
illustrate,  calculate,  use,  interpret,  relate,  manipulate,  apply,  and/or  modify  facts,  concepts,  theories, 
information,  or  data? 

d.  analytical  activities  that  required  me  to  analyze  and  organize  facts,  data,  and  information;  deduce 
patterns  and  trends;  contrast,  compare,  and/or  distinguish  differences  or  similarities;  and  then  discuss 
solutions,  directions  and  plan  or  devise  actions? 

e.  the  synthesis  and  evaluation  of  facts,  information,  data,  situations,  problems,  and  furthermore  require 
me  to  argue  rationally,  support  or  justify  a method,  solution,  action,  choice  of  formula,  theory,  concept, 
principle  or  result  in  the  need  to  propose  a hypothesis,  following  with  the  design  of  an  experiment, 
product,  process,  technique,  and/or  make  judgments  that  had  to  be  critiqued  and  defended  and  finalized 
into  reports,  summaries,  or  papers. 

f.  the  design,  discovery,  invention,  development,  creation,  research,  or  transformation  of  knowledge  into 
products,  processes,  techniques,  models,  methods,  strategies,  etc.,  using  design  and  development, 
research,  experimentation,  and/or  development  knowledge,  techniques,  procedures,  and  tools? 

113.  This  course  engaged  me  in  (Select  one  response) 

a.  learning  knowledge  and  skills  to  use  when  I get  a job. 

b.  learning  knowledge  and  skills  to  use  when  I get  a job,  but  also  provided  the  opportunity  to  apply 

that  knowledge  in  class  through  projects  or  activities  where  performing  tasks  using  that 
knowledge  were  required 

c.  neither  (a)  nor  (b),  very  well 
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114.  The  following  list  identifies  and  briefly  describes  teaching  methods  the  professor  may  use 
during  instruction.  (Select  all  that  apply) 

a.  the  professor  lectures  information  and  connections;  I listen  and  take  notes,  if  I choose 

b.  the  professor  focuses  or  presents  content,  breaks  the  class  into  student  groups  to  discuss  the 
content,  and  then  engages  in  summarizing  and  clarifying  the  content  as  a group. 

c.  the  professor  focuses  or  presents  content  and  then  assigns  individual  but  short  term  projects  using  the 
content  or  information  (e.g.,  problem  to  solve,  design  project,  analysis). 


d.  the  professor  focuses  or  presents  content,  breaks  the  class  into  student  groups  to  discuss  the 

content,  and  then  engages  in  a short  term  group  project  using  the  content  or  information  (e.g.,  problems 
to  solve,  design  project,  analysis) 

e.  lessons  are  broken  down  in  components;  as  individual  students  master  each  component,  they  are 
tested;  when  they  pass  the  test,  they  go  on  to  the  next  component. 

f.  the  professor  uses  visual  charts,  displays,  and/or  a wide  range  of  graphic  organizers  or  other  visuals  to 
better  organize  and  present  information,  to  show  relationships  between  concepts,  principles,  to  increase 
understanding  about  the  application  of  foundation  concepts  or  principles. 

g.  when  presenting  content,  the  professor  uses  examples  that  are  and  are  not  representative  of  the 
concept  or  principle.  Students  compare  the  examples  and  match  those  that  represent  the  concept  or 
not;  gradually,  as  more  examples  that  are  and  are  not  representative  are  reviewed,  the  group  reaches 
consensus  of  what  examples  directly  represent  the  content  and  come  away  with  greater 
understanding. 

h.  lessons  require  that  we  combine  concepts  and  analyze  the  relationships  of  concepts;  we  then  engage 
in  solving  problems. 

i.  during  the  lessons,  the  professor  asks  us  to  identify  and  enumerate  information  related  to  concepts 
as  they  are  demonstrated,  grouping  concepts  into  categories  with  common  attributes. 

j.  we  learn  information  on  concepts  through  the  act  of  classification,  gathering  and  classifying 
information  to  build  and  test  hypotheses;  they  engage  in  experiments  and  the  results  of  experiments  are 
used  to  develop  hypothesis  generalizations  about  the  situation,  idea,  or  problem. 
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115.  The  following  list  identifies  and  briefly  describes  additional  teaching  methods  the  professor 

may  use  during  instruction.  (Select  all  that  apply) 

a.  students  are  presented  with  generalizations  and  examples  and  engage  in  trying  to  identify  the 
individual  situation  or  idea  that  is  embedded  (move  from  problem  to  why  something  happens) 

b.  students  are  presented  with  a problem  and  then  create  questions  to  be  used  to  solve  the  problem. 

Students  engage  in  a process  of  investigation  and  explanation  of  the  phenomena. 

c.  students  engage  in  a formally  organized  court  case  to  present  information  and  arguments  about  the 
ingrained  issues. 

d.  students  are  instructed  on  each  component  of  the  content,  and  all  must  be  successful  on  that  content 
before  the  professor  moves  on  with  new  or  more  complex  content 

e.  lessons  break  skills  down  into  components  and  sequences  of  action;  each  person  learns  the  skill  step  by 
step  the  same  way 

f.  lessons  begin  by  focusing  on  a current  situation;  analogies  are  used  to  define  the  characteristics  of  the 
situation;  analogies  continue,  using  other  graduated  analogies  until  it  appears  to  have  no  relationship  to 
the  origin;  the  lesson  then  uses  the  final  description  of  the  analogy  to  compare  to  the  original  situation 

g.  lessons  engage  us  in  the  development  of  physical  skills,  such  as  welding 

h.  the  professor  uses  metaphors  to  make  content  more  familiar 

i.  lessons  focus  on  personal  development,  free  expression  of  ideas  and  feelings,  furthering  your  self- 
understanding 

j.  students  explore  problems  through  actions  developing  problem  solving  skills;  we  participate  and/or 
observe 
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116.  My  professor  exhibited  the  following  styles  of  instruction  throughout  the  semester. 

(Select  all  that  apply.) 

a.  professor  makes  all  decisions  on  what,  where,  when,  and  how  learning  takes  place; 
is  the  expert;  strives  for  precision,  synchronization,  and  uniformity;  and  determines 
what  is  taught  and  how  it  will  he  evaluated 

h.  students  are  given  a number  of  tasks  to  do  while  in  class;  students  can  ask  questions; 
professor  moves  around  and  gives  feedback 

c.  students  provide  feedback  to  each  other;  one  student  performs  while  another 
provides  feedback;  professor  designs  forms  to  guide  the  observations;  socialization 
is  inherent  in  this  style;  students  develop  feedback  skills 

d.  feedback  is  provided  by  you  as  the  individual  learner  to  yourself;  other  events  providing 
external  feedback  facilitate  your  ability  to  do  this;  professor  helps  you  become  a better 
evaluator,  thus,  increasing  your  self-esteem  about  working  independently 

e.  we  select  our  own  level  of  performance  and  alter  it  according  to  my/our  self-evaluation; 
the  professor  determined  the  tasks  and  defined  the  levels  of  difficulty 

f.  professor  leads  students  to  discover  concept  by  answering  a series  of  questions; 
professor  determines  concepts  and  best  sequences  for  guidance;  friendly  environment  with 
time  to  think  built  into  the  learning  opportunity;  professor  traces  a series  of  questions 
leading  to  the  answer 

g.  professor  presents  question;  students  use  logical  and  critical  thinking  to  discover  solutions; 
students  determine  questions  to  ask  rather  than  the  professor;  professor  respects  the  student 
process  and  does  not  interfere 

h.  professor  encourages  students  to  find  multiple  solutions  to  given  problems;  professor  selects 
the  subject  and  designs  the  problem;  there  is  no  one  right  answer;  professor  responds  to 
student  process  rather  than  the  value  of  a solution  or  answer 

i.  the  student  and  professor  selects  the  content  to  be  learned;  the  student  designs,  develops,  and 
performs  the  series  of  tasks  and/or  students  select  the  activity,  design  the  experiences,  perform 
the  tasks;  professors  assists/consults  with  the  evaluation  of  tasks 

j.  students  are  empowered  to  take  full  responsibility  for  the  learning  process;  they  are  not  required  to 
consult  with  the  professor 
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117.  Which  of  the  following  best  descrihes  this  course? 

Choose  the  one  item  that  comes  closest  to  describing  your  experience  in  this  course. 

a.  The  professor  assumes  the  entire  responsibility  for  delivering  the  course  content.  He/she 
lectures  all  information  we  are  expected  to  learn.  The  text  is  used  as  a reference.  Lectures 
reflect  text  content. 


b.  The  professor  assumes  the  entire  responsibility  for  delivering  the  course  content  in  combination 
with  assigned  readings  from  the  textbook.  The  lectures  and  text  content  provide  all  the  information 
we  are  expected  to  learn.  Most  lectures  correlate  directly  or  are  duplication  of  text  content. 

c.  Students  are  assigned  reading  from  the  text  to  gain  basic  course  content.  My  professor  explains 
difficult  content  from  the  text  and  then  adds  lectures  on  some  important  or  critical  content  that  is 
not  covered  in  the  text,  thus  expanding  or  deepening  understanding  and  ability  to  use  the 
information  from  the  text. 

d.  Students  are  responsible  for  some  of  their  own  learning.  For  example,  once  a concept  or  principle 
is  explained  by  the  professor  and  we  have  used  the  text  for  basic  learning,  as  a source  or 
reference,  we  then  have  to  perform  research  on  content  ourselves  to  deepen  our  understanding  of 
the  concept  and  its  application  possibilities.  We  have  to  bring  the  information  back  to  class  to 
share  with  the  professor  and  class.  Student  activities  can  vary  from  literature  research,  case 
studies,  identifying  additional  sources  of  information  (e.g.,  books,  people,  examples, 
demonstrations,  etc.).  Students  are  required  to  learn  on  their  own  or  in  small  groups  to  deepen 
understanding  or  extend  learning  and  understanding  beyond  that  presented  by  the  professor  or 
established  learning  activities. 

e.  The  professor  assigns  reading  from  the  text,  explains  difficult  content,  and  then  provides  content 
to  deepen  or  extend  the  basic  text  content  or  to  clarify  or  explain  content  not  well  understood. 
Students  are  responsible  for  some  of  their  own  learning,  and  we  then  engage  in  research  to  solidify 
understanding  of  the  content.  Ultimately,  the  professor  then  assigns  projects  that  expand  learning 
into  the  “doing”  dimension  where  we  used  the  content  learned  to  solve  a problem,  develop  a 
product,  construct  a theoretical  model,  use  materials,  processes,  and  knowledge  to  create,  etc. 

f.  Students  are  responsible  for  a great  deal  of  their  own  learning.  After  working  with  us  in  a variety 
of  ways,  many  of  them  are  highly  engaging  students  to  learn  important  knowledge  and  skills, 
where  the  professor  is  more  of  a learning  coach,  direction  setter,  source  of  validation,  someone 
who  models  an  inquiry  driven  process  of  learning,  with  a strong  focus  on  "how"  and  "why" 
processes.  He/she  provides  the  opportunity  to  engage  in  the  creation  of  a solution  to  a identified 
need  or  problem  applying  the  knowledge  and  skills  learned  earlier  or  throughout  the  learning 
processes  throughout  the  semester. 

118.  This  course  provided  the  opportunity  to  work  cooperatively  in  small  groups  to  accomplish  the 

learning  of  course  content.  (Select  one) 

a.  Yes 

b.  No 
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119.  When  working  together,  we  sought  outcomes  that  benefited  me  individually  as  well  as  the 
whole  group.  (Select  one) 

a.  Most  of  the  time 

b.  Some  of  the  time 

c.  Not  really 

c.  No  opportunity  to  work  in  groups 

120.  When  working  with  others,  I feel  that  we  maximized  my  own  learning  and  the  learning  of 
others.  (Select  one) 

a.  Most  of  the  time 

b.  Some  of  the  time 

c.  Not  really 

d.  No  opportunity  to  work  in  groups 

121.  Working  in  groups  provided  greater  opportunity  for  everyone  to  learn  more  and  resulted  in 
higher  grades  for  all.  (Select  one) 

a.  Most  of  the  time 

b.  Some  of  the  time 

d.  Not  really 

e.  No  opportunity  to  work  in  groups 

122.  When  you  were  required  to  work  in  student  groups  throughout  the  course,  were  those  group 
assignments  formally  organized  with  criteria  for  performance?  (Select  one) 

a.  Most  of  the  time 

b.  Some  of  the  time 

c.  Not  really 

d.  No  opportunity  to  work  in  groups 

123.  When  you  were  required  to  work  in  student  groups  throughout  the  course,  did  the  professor 
provide  formal  and  specific  team  related  instruction  on  how  to  function  effectively  and  productively 
on  a team?  (Select  one) 

a.  Yes 

b.  No 
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124.  Working  in  groups  results  in: 

(Select  as  many  as  apply  b-i;  if  you  choose  response  a,  move  on  to  question  125) 

a.  there  was  no  opportunity  to  work  in  groups 

(if  you  choose  this  selection,  move  on  to  question  125) 

b.  higher  achievement  and  productivity  by  all  or  almost  all  members  of  the  group 

c.  longer  term  retention  of  knowledge  being  learned 

d.  intrinsic  (inside  myself)  and  higher  motivation  to  achieve  by  all  or  almost  all  members 
of  the  group;  greater  focus  and  time  on  task 

e.  higher  level  thinking,  reasoning,  deeper  analysis  of  problems,  better  judgments 

f.  more  positive  relationships  between  most  students  or  among  group  members  and  more 
caring  about  each  other’s  learning  and  success;  feelings  of  more  support  in  learning 

g.  greater  value  of  diversity  among  group  members;  greater  cohesion  among  students  in 
the  course 

h.  the  development  of  higher  self-esteem  among  most  students;  further  development  of 
self  identify 

i.  development  of  social  skills  so  that  students  learn  to  engage  with  each  other  in  a positive 
manner,  even  when  conflicting  ideas  are  on  the  table 

j.  greater  ability  to  cope  with  adversity  and  stress 

125.  The  professor's  language  skills  were  not  a barrier  in  communication  between  the  professor 
and  students. 

a.  Strongly  agree  - the  professor's  language  skills  were  exceptionally  good;  very  effective 
communication  took  place  between  the  professor  and  students. 

b.  Agree  - the  professor's  language  skills  were  good;  there  was  effective  communication  between 
the  professor  and  students. 

c.  Disagree  - the  professor's  language  skills  need  to  improve  for  effective  communication  to 
occur  between  the  professor  and  students. 

d.  Strongly  Disagree  - the  professor's  language  skills  were  inadequate  for  effective 
communication  between  the  professor  and  students;  poor  language  skills  resulted  in 
communication  barriers  between  the  professor  and  students. 
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Unlike  Items  101-125  above  which  focused  on  THE  course  you  are  NOW  in  and  completing,  the 
following  questions  are  focused  more  broadly. 

For  Items  126-136,  reflect  on  your  experience  across  ALL  the  courses  you  have  taken  in 
engineering  and/or  technology  to  date.  Provide  your  perspective  by  generalizing  across  ALL  the 
courses  that  you  have  taken  in  engineering  and/or  technology  to  date  and  respond  to  Items  125-135 
below. 

126.  The  professors  teaching  the  engineering  and/or  technology  courses  that  I've  taken  to  date  in 
my  major:  (Select  one) 

a.  seem  exceptionally  competent  and  knowledgeable 

b.  seem  competent  and  knowledgeable 

c.  seem  adequate  in  their  knowledge 

d.  professor's  knowledge  seems  questionable 


127.  The  professors  teaching  the  courses  that  I've  taken  in  engineering  and/or  technology  teach  in  a 

way  that:  (Select  one) 

a.  motivates  me  to  want  to  learn  and  perform  in  those  classes  at  a very  high  level;  they  keep  me 
interested,  excited,  and  make  me  realize  that  I have  chosen  the  right  field  or  career  track  for  me 

b.  keeps  me  interested  most  of  the  time  so  that  I perform  above  average  most  of  the  time 

c.  is  difficult  for  me  to  maintain  my  interest  in  the  courses;  it  is  often  difficult  to  remain  interested  all 
the  way  through  each  class.  I feel  I can  read  the  book  and  take  the  tests  and  still  perform  well  enough 
for  an  adequate  grade 

d.  truly  causes  me  to  be  less  motivated  to  perform,  making  it  almost  impossible  to  remain  interested  in 
the  courses  or  content  being  covered 

128.  The  learning  environment  in  the  college  and  department  is  positive  in  the  following  ways: 
(Select  all  that  apply) 

a.  the  learning  environment  and  climate  are  positive 

b.  there  is  appropriate  technology,  computer  labs,  specialized  technology  related  to  each  discipline 

c.  there  are  good  labs,  lab  equipment, 

d.  there  is  adequate  student  work  space  for  assignments,  projects,  group  meetings,  etc. 

e.  administrators  are  approachable  and  helpful,  e.g.  the  department  chairs  (heads)  and  dean 

f.  faculty  are  available,  approachable,  professional,  and  helpful 

g.  department  and  college  staff  are  available,  professional,  and  helpful  in  solving  problems  or  meeting 
student  needs,  and  friendly 

h.  faculty  take  extra  time,  or  go  the  extra  mile,  and  are  available  to  support  and  assist  students  in 
solving  problems  or  meeting  their  needs 

i the  academic  advising  I have  received  is  of  high  quality  and  accurate 

j.  graduate  teaching  or  lab  assistants  seem  to  be  knowledgeable  and  competent 
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129.  The  learning  environment  in  the  college  and  department  needs  to  improve  the  following: 
(Select  all  that  apply) 

a.  the  learning  environment  and  climate 

b.  technology,  computer  labs,  specialized  technology  related  to  each  discipline 

c.  labs  and  lab  equipment, 

d.  student  work  space  for  assignments,  projects,  group  meetings,  etc. 

e.  administrators  approachability  and  willingness  to  be  helpful(e.  g.,  the  department  chairs  (heads)  and 
dean) 

f.  faculty  availability,  approachability,  professionalism,  and  willingness  to  be  helpful 

g.  department  and  college  staff  are  availability,  professionalism,  and  helpfulness  in  solving  problems  or 
meeting  student  needs,  and  friendliness 

h.  faculty  willingness  to  take  extra  time,  or  go  the  extra  mile,  and  be  available  to  support  and  assist 
students  in  solving  problems  or  meeting  their  needs 

i academic  advising 

j.  knowledge  and  competence  of  graduate  teaching  or  lab  assistants 

130.  Generally,  when  considering  course  Quality,  the  courses  I've  taken  so  far  seem  to  have  had 
well  planned  content,  sound  academic  purpose,  appropriate  and  well  designed  lab  activities,  and 
excellent  execution  of  student  learning  activities  by  the  professor  and/or  grad  assistant. 

(Select  one) 

a.  strongly  agree 

b.  most  or  many  do 

c.  some  (less  than  half)  do 

d.  most  or  many  do  not 

131.  The  courses  that  I've  taken  so  far  seem  to  have  been  well-structured  and  organized  with  clear 
learning  objectives  that  are  focused,  purposeful;  the  courses  have  had  well  designed  and  developed 
syllabi  that  clearly  explain  the  expectations  of  the  professor  for  the  course  and  a schedule  or 
timeline  provides  an  understanding  of  the  events,  due  dates,  and  activities  for  the  semester. 

(Select  one) 

a.  strongly  agree 

b.  most  or  many  do 

c.  some  (less  than  half)  do 

d.  most  or  many  do  not 

For  Items  132-136,  consider  the  connections  between  course  syllabi,  assignments,  and  schedule  for 
all  the  courses  you  taken  to  date;  when  generalizing  across  ALL  the  courses  you  have  taken  in 
engineering  or  technology,  most  of  your  professors:  (Select  one  response  for  each  132-134) 

132.  covered  the  course  content  specified  in  the  syllabus,  expanding  when  appropriate 

a.  yes 

b.  no 

133.  adhered  to  the  assignments  specified  in  the  syllabus  and  didn't  add  anything  significant 

a.  yes 

b.  no 

134.  progressed  through  the  course  according  to  the  schedule  plan  in  the  syllabus 

a.  yes 

b.  no 
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135.  In  most  of  my  courses,  the  tests  (or  other  methods  of  measuring  student  learning  such  as 
projects,  etc.,  papers,  research,  etc.)  are  directly  linked  and  connected  to  the  content  described  in 
the  syllabus.  (Select  one) 

a.  Yes,  most  of  the  time 

b.  Usually,  but  there  are  some  major  deviations  from  the  syllabi  across  courses 

c.  Less  than  half  of  the  time;  there  is  a lot  of  content  on  tests,  or  content  that  we  are  required  to 
know  and  use  for  projects,  etc.  that  was  not  specified  on  course  syllabi 

d.  There  has  often  been  a “dis-connect”  between  the  knowledge  and/or  skills  that  we  were 
tested  on  or  required  to  use  on  projects,  etc.  and  what  was  specified  on  course  syllabi  across  the 
courses  I have  taken 

136.  In  most  of  my  courses,  the  tests  (or  other  methods  of  measuring  student  learning  such  as 
projects,  etc.,  papers,  research,  etc.)  are  directly  linked  and  connected  to  the  content  covered  by 
the  professors.  (Select  one) 

a.  Yes,  most  of  the  time 

b.  Usually,  but  there  have  been  some  major  deviations  by  the  professors  across  courses 

c.  Less  than  half  of  the  time;  there  is  a lot  of  content  on  tests,  or  content  that  we  were  required  to 
know  and  use  for  projects,  etc.  that  was  not  specified  on  course  syllabi  or  covered  by  the 
professors  or  assistants. 

d.  There  has  often  been  a “dis-connect”  between  the  knowledge  and/or  skills  that  we  were 

tested  on  or  required  to  use  on  projects,  etc.  and  what  was  covered  by  the  professors  or  assistants. 
A lot  of  course  content  was  not  covered  by  the  professors  or  assistants. 
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PROFESSORS  — READ  OUT  LOUD  CAREFULLY  to  students  before  handing  out  questionnaires. 


To:  Participating  students 

Fr:  Dean 

Re: 

New  initiative  to  study  the  quality  of  instruction,  course  content,  and  the  learning  environment  across  the 
college 

The  following  questionnaire  is  being  administered  to  selected  classes  across  the  college  and  its  four 
departments.  As  students  in  those  classes,  you  are  being  asked  to  participate  in  providing  baseline  information 
about  the  quality  of  instruction,  course  content,  and  the  learning  environment  in  the  college.  It  is  important  to 
take  note  of  the  following: 

(a)  The  first  part  of  the  questionnaire  relates  only  to  this  course. 

(b)  The  second  part  of  the  questionnaire  relates  to  your  experience  across  all  the  courses  you  have  taken  in  your 
major  department. 

(c)  This  questionnaire  does  not  seek  information  about  your  experience  in  any  courses  outside  the  major 
department  (e.g.  general  education  or  courses  transferred  to  NIU). 

Your  responses  to  these  questions  will  be  used  as  baseline  information  to  study  how  to  strengthen  the  quality  of 
education  across  the  college. 

We  are  hoping  that  you  will  complete  this  questionnaire  thoughtfully,  seriously,  and  genuinely,  with  the 
understanding  that  it  is  important  and  will  assist  us  in  structuring  a college  initiative  to  study  and  strengthen 
the  quality  of  instruction,  course  content,  and  the  learning  environment  across  the  college. 

In  testing  the  questionnaire  with  students,  it  took  about  20  minutes;  therefore,  we  are  allowing  30  minutes  of 
class  time  to  complete  the  questionnaire  in  class. 

The  questionnaire  is  somewhat  long,  but  not  as  long  as  it  may  seem  because  the  questions  have  been  written  in 
a way  that  hopefully  describe  thoroughly  what  we  are  seeking  information  about.  Also  the  print  is  regular  sized, 
and  we  have  spaced  and  printed  the  document  for  easier  reading. 

Please  attend  to  each  item  carefully  and  respond  to  the  best  of  your  ability.  We  need  your  input.  It  is  important 
that  you  respond  honestly,  genuinely,  and  with  sincerity  as  the  results  of  the  survey  will  greatly  impact  the 
Dean's  new  initiative  on  the  quality  of  education  for  students  in  the  college. 

Thank  you  for  investing  your  time  and  serious  effort  to  help  us  begin  this  very  important  initiative. 
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End  of  Course  Questionnaire  on  Teaching  and  Learning 

Jule  Dee  Scarborough,  2006 

After  completing  the  student  and  course  information  on  the  front  side  of  the  scan  form,  respond  to  the  following  questions  on  the 
back  side  of  the  form  beginning  with  item  101. 

Questions  101-124  focus  on  the  course  you  are  now  ending.  Please  respond  to  101-124  based  upon  your  experience 
in  this  course  only. 

101.  The  course  syllabus  identified  specific  learning  objectives. 

2 a.  Yes,  and  I understood  them 
1 b.  Yes,  but  I didn't  understand  them 
0 c.  I don’t  know 

0 d.  No,  there  were  no  learning  objectives 
Max  Points  Possible  — 2. 

102.  The  learning  objectives  for  this  course  were  chosen  or  required  by:  (Select  all  that  apply.) 

1 a.  Future  employers 
0 b.  Department  head 

0 c.  Professor's  interests 

1 d.  Accreditation  agency 

1 e.  NIU  General  Education  Goals 
Of.  I don't  know 
Max  Points  Possible  — 3 

103.  The  course  syllabus  specified:  (Select  all  that  apply) 

1 a.  course  or  student  learning  objectives 

1 b.  course  description 

1 c.  clearly  defined  course  content 

1 d.  clearly  defined  assignments,  labs,  papers,  projects,  tests,  or  other  important  assignments  or  activities 

1 e.  the  course  schedule  or  timeline  identifying  meeting  dates,  assignment  due  dates,  and  the  semester's  schedule 

1 f.  additional  explanations  of  course  requirements  that  established  the  criteria  for  each  assignment 

1 g.  references  other  than  the  text,  e.g.  books,  websites,  articles,  other  sources  related  to  course  content 

1 h.  contact  information  for  professor,  instructor,  and/or  graduate  teaching  or  lab  assistants 

Max  Points  Possible  — 8 

104.  The  professor  (and  any  assistants):  (Select  all  that  apply) 

1 a.  focused  content  and  learning  activities  on  the  course  or  student  learning  objectives  throughout  the  semester 
1 b.  provided  learning  that  seemed  to  align  with  the  course  description 
1 c.  taught  the  course  content  specified  in  the  syllabus 

1 d.  followed  the  assignments,  labs,  papers,  projects,  tests,  or  other  important  assignments  or  activities 
outlined  and  defined  in  the  syllabus 

1 e.  followed  the  course  schedule  or  timeline  specified  in  the  syllabus  (e.g.,  meeting  dates,  assignment  due  dates,  and 
the  semester's  schedule) 

1 f.  graded  assignments  according  to  the  written  explanations  for  course  requirements  establishing  the  criteria  for  each 
assignment 

1 g.  was(were)  available,  using  the  contact  information  for  professor,  instructor,  and/or  graduate  assistants 
1 h.  deviated  from  the  syllabus  by  adding  appropriate  content  to  expand,  deepen  understanding,  or  resolve  questions 
resulting  in  adding  value  to  the  course;  any  additional  assignments  were  appropriate  having  reasonable  timelines 
-1  i.  deviated  from  the  syllabus  inappropriately  where  additions  to  the  information  provided  on  the  syllabus,  or  new 
assignments  added,  were  irrelevant  or  distracting  and  added  little  or  no  value  to  the  course  or  learning;  new 
assignments  were  untimely  and  caused  unnecessary  stress  for  students 
1 j.  The  course  was  well  organized,  structured,  and  executed. 

Max  Points  Possible  — 8 
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105.  Which  of  the  following  methods  were  used  hy  the  professor  to  measure  learning?  (Select  all  that  apply) 

1  a.  final  exam  - traditional  test 
1 b.  midterm  exam  - traditional  test 

1 c.  quizzes  and/or  short  tests  periodically-traditional  test(s)  (e.g.,  multiple-choice  or  true/false) 

1 d.  quizzes  and/or  short  tests  periodically  - short  answer  and/or  essay 

1 e.  research  or  learning  paper  (s),  usually  requiring  literature  search  or  field  research  and  formal  write-up 
1 f.  case  study(ies)  in  industry,  usually  requiring  a report  or  short  paper  write-up 
1 g.  hands-on  technical  project(s) 

1 h.  hands-on  non-technical  project(s) 

1 i.  other  types  of  performances,  "doing"  something 

1 j.  course  portfolio,  full  documentation  of  all  work  and  progress  in  the  course 
k.  other;  write  a description  here: 

Max  Points  Possible  — 10 

106.  Select  ALL  the  descriptions  helow  that  identify  the  methods  being  used  in  this  course  to  measure  student 
learning: 

1 a.  Learning  was  measured  on  my  ability  to  memorize  terminology,  symbols,  facts,  information,  theory, 

principles,  concepts,  information,  definitions,  descriptions 

2 b.  Learning  was  measured  on  my  ability  to  make  comparisons  to  determine  similar  and  dissimilar 

examples,  understanding  relationships  and  connections  between  and  among  facts,  concepts,  theories, 
principles,  translates  knowledge  into  a new  context,  interpret  facts,  predict  consequences,  order,  group 
information,  contrast,  distinguish,  estimate,  differentiate,  discuss,  or  extend  knowledge 

3 c.  Learning  was  measured  on  my  ability  to  use  information,  methods,  concepts,  theories  in  new  situations; 

problem  solving  - this  requires  choosing  and  applying  knowledge  (e.g.,  the  best  formula,  concept, 
principle,  theory  to  solve  problems),  using  inductive  reasoning  to  determine  the  best  methods, 
techniques,  tools,  strategies  to  apply  towards  a best  solution;  this  method  of  measurement  can  range 
from  a test  item  with  a complex  problem  to  be  solved  or  a hands-on  technical  problem  requiring  the 
design  and  building  of  something  mechanical.  The  key  to  this  method  is  that  it  requires  application  of 
knowledge  - "doing"  (demonstrate,  calculate,  illustrate,  show,  solve,  examine,  modify,  relate,  change, 
experiment,  discover). 

4 d.  Learning  was  measured  on  my  ability  to  recognize  patterns  in  information,  problems,  and  situations; 

the  ability  to  organize  parts,  identify  or  discover  "hidden"  meanings,  and/or  identify  components;  this 
requires  one  to  analyze,  separate  thoughts,  processes,  problems,  order,  explain,  connect,  classify,  and  divide, 
compare,  select,  explain,  and/or  make  inferences  (indirect  meanings);  this  requires  deductive  reasoning  where 
one  begins  with  facts  and  information,  makes  choices  to  gradually  discover  the  bigger  picture 

5 e.  Learning  measured  my  ability  to  hypothesize,  design,  support  argument,  schematize,  write,  report, 

justify,  choose,  evaluate,  estimate,  judge,  criticize,  defend,  use  old  ideas  to  create  new  ones,  extending 
the  old  idea  into  a new  one  for  extended  applications,  make  choices  based  upon  reasoned  argument, 
verify  value  of  evidence,  recognize  when  subjectivity  is  being  used  rather  than  objectivity  (more 
scientific),  make  sound  generalizations  from  given  facts,  relate  and  use  knowledge  across  different 
contexts,  predict  and  draw  conclusions,  combine,  integrate,  modify,  rearrange,  substitute  knowledge, 
plan,  formulate,  compare  and  discriminate  between,  summarize,  and  make  conclusions 

6 f.  Learning  measured  my  ability  to  design,  discover,  invent,  develop,  create,  research;  transform 

knowledge  into  a product,  process,  technique,  model,  method,  strategy,  etc. 

Points  for  only  highest  level  response  only.  Max  Points  Possible  - 6 
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107.  Select  the  response  that  best  describes  the  relationship  between  the  traditional  tests  you  have  taken  to  date  in 
this  course  (e.g.  multiple-choice,  true/false  items,  etc.)  and  the  course  content . 

3 a.  the  content  of  the  test(s)  was  related  to  the  content  specified  in  the  syllahus,  and  only  to  content  specified  in  the 
syllahus. 

2 h.  the  content  of  the  test(s)  was  related  to  the  content  specified  in  the  syllahus  and  other  content  proyided  hy  the 
professor  or  assistants. 

1 c.  the  content  of  the  test(s)  did  not  relate  to  the  content  specified  in  the  syllahus  but  did  relate  the  other  content 
proyided  hy  the  professor  or  assistants. 

0 d.  the  content  of  the  test(s)  related  to  neither  (1)  the  content  specified  in  the  syllahus.  nor  (2)  the  other  content 
proyided  hy  the  professor  or  assistants. 

Max  Points  Possible  — 3 

Items  108-111  relate  to  the  measurement  of  student  learning  through  performance(s)  rather  than  traditional  tests. 
***  Consider  the  definitions  below  when  responding  to  items  108-111. 

***  Definitions: 

***Performance  Task  (or  assessment)  - any  authentic  or  real-world  task  designed  to  measure  student  learning.  Such  a 
task  can  be  used  to  determine  what  students  can  "do"  with  knowledge.  Unlike  some  traditional  tests,  performance  tasks 
require  students  to  move  to  another  level  of  providing  evidence  of  learning  - that  of  applying  or  using  knowledge  by 
performing  authentic  tasks,  such  as  designing  a part  or  product,  or  designing  and  then  producing  the  part  or  product. 
Writing  a paper  would  provide  evidence  of  research  skills  and  communication  skills,  for  example. 

(108)  Performance  tasks  were  used  to  measure  student  learning  in  this  course.  (*see  definition  above) 

la.  Yes  (according  to  the  definition  above) 

0 h.  No  (according  to  the  definition  above) 

Max  Points  Possible  — 1 

***Rubric  - any  type  of  information  sheet  or  form,  check  off  sheet  that  establishes  the  levels  of  performance  criteria  for 
performance  tasks;  these  criteria  establish  standards  for  performance  and  the  criteria  for  each  standard.  They  are  used  to 
provide  students  information  about  what  is  required  to  achieve  a particular  number  of  points  or  grade.  See  attached 
example  at  end  of  questionnaire  following  this  page;  then  continue  to  complete  the  questionnaire.. 

(109)  Rubrics  were  used  for  scoring  or  grading  the  performances  in  this  course. 

la.  Yes  (according  to  the  definition  below) 

0 b.  No  (according  to  the  definition  below) 

Max  Points  Possible  — 1 

(110)  Below  are  examples  of  some  performance  tasks;  identify  any  that  are  similar  to  performances  that  you  had 
to  accomplish  during  this  course.  Select  all  that  apply: 

1 a.  Writing  a paper 

1 b.  Working  problems,  showing  the  entire  equation  worked  out  manually,  through  each  step  of  the  equation 
1 c.  Designing  a product  part,  entire  machine,  other  major  design  project 
1 d.  Designing  an  industrial  production  system 
1 e.  Designing  electrical  circuitry  or  full  electrical/electronic  system 

1 f.  Designing  and  producing  a part  using  manufacturing  processes,  e.g.  actually  producing  product  using 
manufacturing  production  equipment  in  a lab  or  on-site  in  industry 
Max  Points  Possible  — 6 
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(Ill)  Select  all  examples  of  performance  tasks  below  (similar)  where  a rubric  or  performance  criteria  form  was 
used  to  score  or  grade  the  performance(s)  during  this  course. 

1 a.  Writing  a paper 

1 b.  Working  problems,  showing  the  entire  equation  worked  out  manually,  through  each  step  of  the  equation 
1 c.  Designing  a product  part,  entire  machine,  other  major  design  project 
1 d.  Designing  an  industrial  production  system 
1 e.  Designing  electrical  circuitry  or  full  electrical/electronic  system 

1 f.  Designing  and  producing  a part  using  manufacturing  processes,  e.g.  actually  producing  product  using 
manufacturing  production  equipment  in  a lab  or  on-site  in  industry 
Max  Points  Possible  — 6 

112.  The  following  items  related  to  levels  of  learning  and  how  learning  takes  place. 

(Select  ALL  that  apply) 

1 a.  the  learning  of  basic  knowledge  requiring  me  to  list,  name,  identify,  show,  define,  recognize,  recall,  state, 
visualize,  state  facts,  concepts,  theories,  principles,  and/or  information? 

2 b.  the  comprehension  or  greater  understanding  of  knowledge  through  activities  that  required  me  to 
summarize,  explain,  interpret,  describe,  compare,  paraphrase,  differentiate  , demonstrate,  classify,  or  contrast 
facts,  information,  concepts,  theories,  principles? 

3 c.  the  application  or  opportunity  to  “do”  or  “perform,”  using  knowledge,  requiring  me  to  solve  problems, 
illustrate,  calculate,  use,  interpret,  relate,  manipulate,  apply,  modify  facts,  concepts,  theories,  information,  or 
data? 

4 d.  analytical  activities  that  required  me  to  analyze  and  organize  facts,  data,  and  information;  deduce  patterns, 
and  trends;  contrast,  compare,  distinguish,  differences  or  similarities;  and  then  discuss  solutions,  directions  and 
plan  or  devise  actions? 

5 e.  the  synthesis  and  evaluation  of  facts,  information,  data,  situations,  problems,  and  furthermore  require  me 
to  argue  rationally,  support  or  justify  a method,  solution,  action,  choice  of  formula,  theory,  concept,  principle  or 
result  in  the  need  to  propose  a hypothesis,  following  with  the  design  of  an  experiment,  product,  process, 
technique,  and/or  make  judgments  that  had  to  be  critiqued  and  defended  and  finalized  into  reports,  summaries, 
or  papers. 

6 f.  the  design,  discovery,  invention,  development,  creation,  research,  or  transformation  of  knowledge  into 
products,  processes,  techniques,  models,  methods,  strategies,  etc.,  using  design  and  development,  research, 
experimentation,  and/or  development  knowledge,  techniques,  procedures,  and  tools? 

Points  for  highest  level  only.  Max  Points  Possible  - 6 

113.  This  course  engaged  me  in  (Select  one  response) 

0 a.  learning  knowledge  and  skills  to  use  when  I get  a job. 

1 b.  learning  knowledge  and  skills  to  use  when  I get  a job,  but  also  provided  the  opportunity  to  apply 

that  knowledge  in  class  through  projects  or  activities  where  performing  tasks  using  that 
knowledge  were  required 
0 c.  neither  (a)  nor  (b),  very  well 
Max  Points  Possible  — 1 
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114.  The  following  list  identifies  and  briefly  describes  teaching  methods  the  professor  may  use  during 

instruction.  (Select  all  that  apply) 

1 a.  the  professor  lectures  information  and  connections;  I listen  and  take  notes,  if  I choose 

1 b.  the  professor  focuses  or  presents  content,  then  breaks  the  class  into  student  groups  to  discuss  the 
content,  then  engages  in  summarizing  and  clarifying  the  content  as  a group. 

1 c.  the  professor  focuses  or  presents  content,  then  assigns  individual  but  short  term  projects  using  the 
content  or  information,  e.g.  problem  to  solve,  design  project,  analysis. 

1 d.  the  professor  focuses  or  presents  content,  breaks  the  class  into  student  groups  to  discuss  the 

content,  and  then  engages  in  a short  term  group  project  using  the  content  or  information  (e.g.,  problems  to 
solve,  design  project,  analysis) 

1 e.  lessons  are  broken  down  in  components;  as  individual  students  master  each  component,  they  are 
tested.  When  they  pass  the  test,  they  go  on  to  the  next  component. 

1 f.  the  professor  uses  visual  charts,  displays,  a wide  range  of  graphic  organizers  or  other  visuals  to 

better  organize  and  present  information;  to  show  relationships  between  concepts  and  principles;  and  to 
increase  understanding  about  the  application  of  foundation  concepts  or  principles. 

1 g.  when  presenting  content,  the  professor  uses  examples  that  are  and  are  not  representative  of  the 
concept  or  principle.  Students  compare  the  examples  and  match  those  that  represent  the  concept  or 
not;  gradually  as  more  examples  that  are  and  are  not  representative  are  reviewed,  the  group  reaches 
consensus  of  what  examples  directly  represent  the  content  and  come  away  with  greater 
understanding. 

1 h.  lessons  require  that  we  combine  concepts  and  analyze  the  relationships  of  concepts;  we  then  engage 
in  solving  problems. 

1 i.  during  the  lessons,  the  professor  asks  us  to  identify  and  enumerate  information  related  to  concepts 
as  they  are  demonstrated,  grouping  concepts  into  categories  with  common  attributes. 

1 j.  we  learn  information  on  concepts  through  the  act  of  classification,  gathering  and  classifying 

information  to  build  and  test  hypotheses;  they  engage  in  experiments  and  the  results  of  experiments  are 
used  to  develop  hypothesis  generalizations  about  the  situation,  idea,  or  problem. 

Max  Points  Possible  = 10 
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115.  The  following  list  identifies  and  briefly  describes  additional  teaching  methods  the  professor  may  use 

during  instruction.  (Select  all  that  apply) 

1 a.  students  are  presented  with  generalizations  and  examples  and  engage  in  trying  to  identify  the 
individual  situation  or  idea  that  is  embedded  (move  from  problem  to  why  something  happens) 

1 b.  students  are  presented  with  a problem  and  then  create  questions  to  be  used  to  solve  the  problem.  Students 
engage  in  a process  of  investigation  and  explanation  of  the  phenomena. 

1 c.  students  engage  in  a formally  organized  court  case  to  present  information  and  arguments  about  the 
ingrained  issues. 

1 d.  students  are  instructed  on  each  component  of  the  content,  and  all  must  be  successful  on  that  content 
before  the  professor  moves  on  with  new  or  more  complex  content 

1 e.  lessons  break  skills  down  into  components  and  sequences  of  action;  each  person  learns  the  skill  step  by 
step  the  same  way 

1 f.  lessons  begin  by  focusing  on  a current  situation;  analogies  are  used  to  define  the  characteristics  of  the 
situation;  analogies  continue,  using  other  graduated  analogies  until  it  appears  to  have  no  relationship  to 
the  origin;  the  lesson  then  uses  the  final  description  of  the  analogy  to  compare  to  the  original  situation 

1 g.  lessons  engage  us  in  the  development  of  physical  skills,  such  as  welding 

1 h.  the  professor  uses  metaphors  to  make  content  more  familiar 

1 i.  lessons  focus  on  personal  development,  free  expression  of  ideas  and  feelings,  furthering  your  self- 
understanding 

1 j.  students  explore  problems  through  actions  developing  problem  solving  skills;  we  participate  and/or 
observe 

Max  Points  Possible  = 10 
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116.  My  professor  exhibited  the  following  styles  of  instruction  throughout  the  semester.  (Select  all  that  apply.) 

1 a.  professor  makes  all  decisions  on  what,  where,  when,  and  how  learning  takes  place; 

is  the  expert;  strives  for  precision,  synchronization,  and  uniformity;  determines 
what  is  taught  and  how  it  will  he  evaluated 

1 h.  students  are  given  a number  of  tasks  to  do  while  in  class;  students  can  ask  questions; 

professor  moves  around  and  gives  feedback 

1 c.  students  provide  feedback  to  each  other;  one  student  performs  while  another 

provides  feedback;  professor  designs  forms  to  guide  the  observations;  socialization 
is  inherent  in  this  style;  students  develop  feedback  skills 

1 d.  feedback  is  provided  by  you  as  the  individual  learner  to  yourself;  other  events  providing 
external  feedback  facilitate  your  ability  to  do  this;  professor  helps  you  become  a better 
evaluator,  thus,  increasing  your  self-esteem  about  working  independently 

1 e.  we  select  our  own  level  of  performance  and  alter  it  according  to  my/our  self-evaluation; 

the  professor  determined  the  tasks  and  defined  the  levels  of  difficulty 

1 f.  professor  leads  students  to  discover  concept  by  answering  a series  of  questions; 

professor  determines  concepts  and  best  sequences  for  guidance;  friendly  environment  with 
time  to  think  built  into  the  learning  opportunity;  professor  traces  a series  of  questions  leading 
to  the  answer 

1 g.  professor  presents  question;  students  use  logical  and  critical  thinking  to  discover  solutions; 

students  determine  questions  to  ask  rather  than  the  professor;  professor  respects  the  student 
process  and  dos  not  interfere 

1 h.  professor  encourages  students  to  find  multiple  solutions  to  given  problems;  professor  selects 

the  subject  and  designs  the  problem;  there  is  no  one  right  answer;  professor  responds  to 
student  process  rather  than  the  value  of  a solution  or  answer 

1 i.  the  student  and  professor  select  the  content  to  be  learned;  the  student  designs,  develops,  and 

performs  the  series  of  tasks  and/or  students  select  the  activity,  design  the  experiences,  perform 
the  tasks;  professors  assists/consults  with  the  evaluation  of  tasks 

1 j.  students  are  empowered  to  take  full  responsibility  for  the  learning  process;  they  are  not  required  to 

consult  with  the  professor 
Max  Points  Possible  — 10 
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117.  Which  of  the  following  best  descrihes  this  course? 

Choose  the  one  item  that  comes  closest  to  describing  your  experience  in  this  course. 

0  a.  The  professor  assumes  the  entire  responsibility  for  delivering  the  course  content.  He/she 
lectures  all  information  we  are  expected  to  learn.  The  text  is  used  as  a reference.  Lectures 
reflect  text  content. 

0 b.  The  professor  assumes  the  entire  responsibility  for  delivering  the  course  content  in  combination 

with  assigned  readings  from  the  textbook.  The  lectures  and  text  content  provide  all  the  information 
we  are  expected  to  learn.  Most  lectures  correlate  directly  or  are  duplication  of  text  content. 

0 c.  Students  are  assigned  reading  from  the  text  to  gain  basic  course  content.  My  professor  explains 

difficult  content  from  the  text  and  then  adds  lectures  on  some  important  or  critical  content  that  is  not 
covered  in  the  text,  thus  expanding  or  deepening  understanding  and  ability  to  use  the  information 
from  the  text. 

1 d.  Students  are  responsible  for  some  of  their  own  learning.  For  example,  once  a concept  or  principle  is 

explained  by  the  professor  and  we  have  used  the  text  for  basic  learning,  as  a source  or  reference,  we 
then  have  to  perform  research  on  content  ourselves  to  deepen  our  understanding  of  the  concept  and  its 
application  possibilities.  We  have  to  bring  the  information  back  to  class  to  share  with  the  professor  and 
class.  Student  activities  can  vary  from  literature  research,  case  studies,  identifying  additional  sources  of 
information,  e.g.  books,  people,  examples,  demonstrations,  etc.  Students  are  required  to  learn  on  their 
own  or  in  small  groups  to  deepen  understanding  or  extend  learning  and  understanding  beyond  that 
presented  by  the  professor  or  established  learning  activities. 

2 e.  The  professor  assigns  reading  from  the  text,  explains  difficult  content,  and  then  provides  content  to 

deepen  or  extend  the  basic  text  content  or  to  clarify  or  explain  content  not  well  understood.  Students  are 
responsible  for  some  of  their  own  learning,  and  we  then  engage  in  research  to  solidify  our  understanding 
of  the  content.  Ultimately,  the  professor  then  assigns  projects  that  expand  learning  into  the  “doing” 
dimension  where  we  used  the  content  learned  to  solve  a problem,  develop  a product,  construct  a 
theoretical  model,  use  materials,  processes,  and  knowledge  to  create,  etc. 

3 f.  Students  are  responsible  for  a great  deal  of  their  own  learning.  After  working  with  us  in  a variety  of 

ways,  many  of  them  are  highly  engaging  students  to  learn  important  knowledge  and  skills  where  the 
professor  is  more  of  a learning  coach,  direction  setter,  source  of  validation,  someone  who  models  an 
inquiry  driven  process  of  learning,  with  a strong  focus  on  "how"  and  "why"  processes.  He/she 
provides  the  opportunity  to  engage  in  the  creation  of  a solution  to  an  identified  need  or  problem, 
applying  the  knowledge  and  skills  learned  earlier  or  throughout  the  learning  processes  throughout  the 
semester. 

Max  Points  Possible  = 3 

118.  This  course  provided  the  opportunity  to  work  cooperatively  in  small  groups  to  accomplish  the 
learning  of  course  content.  (Select  one) 

1 a.  Yes 
0 b.  No 

Max  Points  Possible  = 1 
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119.  When  working  together,  we  sought  outcomes  that  benefited  me  individually  as  well  as  the 
whole  group.  (Select  one) 

2 a.  Most  of  the  time 
1 b.  Some  of  the  time 
0 c.  Not  really 

0  c.  No  opportunity  to  work  in  groups 
Max  Points  Possible  = 2 


120.  When  working  with  others,  I feel  that  we  maximized  my  own  learning  and  the  learning  of  others. 
(Select  one) 

2 a.  Most  of  the  time 

1 b.  Some  of  the  time 
0 c.  Not  really 

0 d.  No  opportunity  to  work  in  groups 
Max  Points  Possible  = 2 

121.  Working  in  groups  provided  greater  opportunity  for  everyone  to  learn  more  and  resulted  in  higher 
grades  for  all.  (Select  one) 

2 a.  Most  of  the  time 

1 b.  Some  of  the  time 
0 c.  Not  really 

0 d.  No  opportunity  to  work  in  groups 
Max  Points  Possible  = 2 

122.  When  you  were  required  to  work  in  student  groups  throughout  the  course,  were  those  group 
assignments  formally  organized  with  criteria  for  performance?  (Select  one) 

2 a.  Most  of  the  time 

1 b.  Some  of  the  time 
0 c.  Not  really 

0 d.  No  opportunity  to  work  in  groups 
Max  Points  Possible  = 2 

123.  When  you  were  required  to  work  in  student  groups  throughout  the  course,  did  the  professor  provide 
formal  and  specific  team  related  instruction  on  how  to  function  effectively  and  productively  on  a team? 
(Select  one) 

1 a.  Yes 
0 b.  No 

Max  Points  Possible  = 1 
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124.  Working  in  groups  results  in: 

(Select  as  many  as  apply  b-i;  if  you  choose  response  a,  move  on  to  question  125) 
0 a.  there  was  no  opportunity  to  work  in  groups 
(if  you  choose  this  selection,  move  on  to  question  125) 


1 b.  higher  achievement  and  productivity  by  all  or  almost  all  members  of  the  group 

1 c.  longer  term  retention  of  knowledge  being  learned 

1 d.  intrinsic  (inside  myself)  and  higher  motivation  to  achieve  by  all  or  almost  all  members 
of  the  group;  greater  focus  and  time  on  task 

1 e.  higher  level  thinking,  reasoning,  deeper  analysis  of  problems,  better  judgments 

1 f.  more  positive  relationships  between  most  students  or  among  group  members  and  more 
caring  about  each  other’s  learning  and  success;  feelings  of  more  support  in  learning 

1 g.  greater  value  of  diversity  among  group  members;  greater  cohesion  among  students  in 
the  course 

1 h.  the  development  of  higher  self-esteem  among  most  students;  further  development  of 
self  identify 

1 i.  development  of  social  skills  so  that  students  learn  to  engage  with  each  other  in  a positive 
manner,  even  when  conflicting  ideas  are  on  the  table 

1 j.  greater  ability  to  cope  with  adversity  and  stress 

Max  Points  Possible  = 9 

125.  The  professor's  language  skills  were  not  a barrier  in  communication  between  the  professor 

and  students. 

4 a.  Strongly  agree  - the  professor's  language  skills  were  exceptionally  good;  very  effective 
communication  took  place  between  the  professor  and  students. 

3 b.  Agree  - the  professor's  language  skills  were  good;  there  was  effective  communication  between  the 
professor  and  students. 

1 c.  Disagree  - the  professor's  language  skills  need  to  improve  for  effective  communication  to 
occur  between  the  professor  and  students. 

0 d.  Strongly  Disagree  - the  professor's  language  skills  were  inadequate  for  effective  communication 
between  the  professor  and  students;  poor  language  skills  resulted  in  communication  barrier 
between  the  professor  and  students. 

Max  Points  Possible  = 4 
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Unlike  Items  101-125  above  which  focused  on  THE  course  you  are  NOW  in  and 
completing,  the  followins  questions  are  focused  more  broadly. 

For  Items  126-136,  reflect  on  your  experience  across  ALL  the  courses  you  have  taken 
in  engineering  and/or  technology  to  date.  Provide  your  perspective  by  seneralizins  across 
ALL  the  courses  that  you  have  taken  in  engineering  and/or  technology  to  date  and  respond  to 
Items  125-135  below. 

126.  The  professors  teaching  the  engineering  and/or  technology  courses  that  I've  taken  to  date  in  my 
major:  (Select  one) 

3 a.  seem  exceptionally  competent  and  knowledgeable 
2 b.  seem  competent  and  knowledgeable 
1 c.  seem  adequate  in  their  knowledge 

0 d.  professor's  knowledge  seems  questionable 
Max  Points  Possible  = 3 

127.  The  professors  teaching  the  courses  that  I've  taken  in  engineering  and/or  technology  teach  in  a way 
that:  (Select  one) 

2 a.  motivates  me  to  want  to  learn  and  perform  in  those  classes  at  a very  high  level;  they  keep  me  interested, 
excited,  and  make  me  realize  that  I have  chosen  the  right  field  or  career  track  for  me 

1 b.  keeps  me  interested  most  of  the  time  so  that  I perform  above  average  most  of  the  time 

0 c.  is  difficult  for  me  to  maintain  my  interest  in  the  courses;  it  is  often  difficult  to  remain  interested  all 
the  way  through  each  class;  I feel  I can  read  the  book  and  take  the  tests  and  still  perform  well  enough 
for  an  adequate  grade 

0 d.  truly  causes  me  to  be  less  motivated  to  perform,  making  it  almost  impossible  to  remain  interested  in 
the  courses  or  content  being  covered 
Max  Points  Possible  = 2 

128.  The  learning  environment  in  the  college  and  department  is  positive  in  the  following  ways: 

(Select  all  that  apply) 

1 a.  the  learning  environment  and  climate  are  positive 

1 b.  there  is  appropriate  technology,  computer  labs,  specialized  technology  related  to  each  discipline 
1 c.  there  are  good  labs,  lab  equipment, 

1 d.  there  is  adequate  student  work  space  for  assignments,  projects,  group  meetings,  etc. 

1 e.  administrators  are  approachable  and  helpful  (e.g.,  the  department  chairs  (heads)  and  dean) 

1 f.  faculty  are  available,  approachable,  professional,  and  helpful 

1 g.  department  and  college  staff  are  available,  professional,  and  helpful  in  solving  problems  or 
meeting  student  needs,  and  friendly 

1 h.  faculty  take  extra  time,  or  go  the  extra  mile,  and  are  available  to  support  and  assist  students  in 
solving  problems  or  meeting  their  needs 
1 i the  academic  advising  I have  received  is  of  high  quality  and  accurate 
1 j.  graduate  teaching  or  lab  assistants  seem  to  be  knowledgeable  and  competent 
Max  Points  Possible  = 10 
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129.  The  learning  environment  in  the  college  and  department  needs  to  improve  the  following: 

(Select  all  that  apply) 

-1  a.  the  learning  environment  and  climate 

-1  b.  technology,  computer  labs,  specialized  technology  related  to  each  discipline 
-1  c.  labs  and  lab  equipment, 

-1  d.  student  work  space  for  assignments,  projects,  group  meetings,  etc. 

-1  e.  administrators  approachability  and  willingness  to  be  helpful  (e.g.,  the  department  chairs  (heads) 
and  dean) 

-1  f.  faculty  availability,  approachability,  professionalism,  and  willingness  to  be  helpful 
-1  g.  department  and  college  staff  are  availability,  professionalism,  and  helpfulness  in  solving 
problems  or  meeting  student  needs,  and  friendliness 
-1  h.  faculty  willingness  to  take  extra  time,  or  go  the  extra  mile,  and  be  available  to  support  and  assist 
students  in  solving  problems  or  meeting  their  needs 
-1  i academic  advising 

-1  j.  knowledge  and  competence  of  graduate  teaching  or  lab  assistants 
Max  Points  Possible  = 0 

130.  Generally,  when  considering  course  quality,  the  courses  I've  taken  so  far  seem  to  have  had  well 
planned  content,  sound  academic  purpose,  appropriate  and  well  designed  lah  activities,  and  excellent 
execution  of  student  learning  activities  hy  the  professor  and/or  grad  assistant. 

(Select  one) 

3 a.  strongly  agree 

2 b.  most  or  many  do 

1 c.  some  (less  than  half)  do 

0 d.  most  or  many  do  not 
Max  Points  Possible  = 3 

131.  The  courses  that  I've  taken  so  far  seem  to  have  been  well-structured  and  organized  with  clear 
learning  objectives  that  are  focused,  purposeful;  the  courses  have  had  well  designed  and  developed 
syllabi  that  clearly  explain  the  expectations  of  the  professor  for  the  course  and  a schedule  or  timeline 
provides  an  understanding  of  the  events,  due  dates,  and  activities  for  the  semester. 

(Select  one) 

3 a.  strongly  agree 

2 b.  most  or  many  do 

1 c.  some  (less  than  half)  do 
0 d.  most  or  many  do  not 
Max  Points  Possible  = 3 
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For  Items  132-136,  consider  the  connections  between  course  syllabi,  assignments,  and  schedule  for  all  the 
courses  you  taken  to  date;  when  generalizing  across  ALL  the  courses  you  have  taken  in  engineering  or 
technology,  most  of  your  professors: 

(Select  one  response  for  each  132-134) 

132.  covered  the  course  content  specified  in  the  syllabus,  expanding  when  appropriate 

1 a.  yes 
0 b.  no 

Max  Points  Possible  = 1 


133.  adhered  to  the  assignments  specified  in  the  syllabus  and  didn't  add  anything  significant 

1 a.  yes 

0 b.  no 

Max  Points  Possible  =1 

134.  progressed  through  the  course  according  to  the  schedule  plan  in  the  syllabus 

1 a.  yes 
0 b.  no 

Max  Points  Possible  = 1 

135.  In  most  of  my  courses,  the  tests  (or  other  methods  of  measuring  student  learning  such  as  projects, 
etc.,  papers,  research,  etc.)  are  directly  linked  and  connected  to  the  content  described  in  the  syllabus. 
(Select  one) 

2 a.  Yes,  most  of  the  time 

1 b.  Usually,  but  there  are  some  major  deviations  from  the  syllabi  across  courses 

0 c.  Less  than  half  of  the  time;  there  is  a lot  of  content  on  tests,  or  content  that  we  are  required  to 

know  and  use  for  projects,  etc.  that  was  not  specified  on  course  syllabi 
-Id.  There  has  often  been  a “dis-connect”  between  the  knowledge  and/or  skills  that  we  were 

tested  on  or  required  to  use  on  projects,  etc.  and  what  was  specified  on  course  syllabi  across  the 
courses  I have  taken 
Max  Points  Possible  = 2 

136.  In  most  of  my  courses,  the  tests  (or  other  methods  of  measuring  student  learning  such  as  projects, 
etc.,  papers,  research,  etc.)  are  directly  linked  and  connected  to  the  content  covered  by  the  professors. 
(Select  one) 

2 a.  Yes,  most  of  the  time 

1 b.  Usually,  but  there  have  been  some  major  deviations  by  the  professors  across  courses 

0 c.  Less  than  half  of  the  time;  there  is  a lot  of  content  on  tests  or  content  that  we  were  required  to 
know  and  use  for  projects,  etc.  that  was  not  specified  on  course  syllabi  or  covered  by  the 
professors  or  assistants. 

-Id.  There  has  often  been  a “dis-connect”  between  the  knowledge  and/or  skills  that  we  were 

tested  on  or  required  to  use  on  projects,  etc.  and  what  was  covered  by  the  professors  or  assistants. 
A lot  of  course  content  was  not  covered  by  the  professors  or  assistants. 

Max  Points  Possible  = 2 


14 


Assessment  - Orientation 


1.  Describe  the  "national  call  for  action"  regarding  teaching. 

2.  Identify  and  describe  the  four  types  of  research  defined  by  Ernest  Boyer  (1990). 

3.  Describe  the  "reversed"  instructional  design  process  - simply  describe.  Where  are  we  in  the  "reverse"  process? 

4.  Describe  a learning  community.  What  are  learning  circles? 

5.  What  are  its  primary  characteristics? 

6.  What  are  communities  of  practice? 

7.  Identify  the  components  of  Deming's  quality  improvement  cycle.  Where  are  we  in  that  cycle? 

8.  Describe  Senge's  meaning  of  "learning"  for  individuals  and/or  organizations.  What  would  you  consider  to  be  important 
aspects  of  his  definition  of  learning? 

9.  Describe  your  "duty"  as  a professor. 

10.  What  are  your  responsibilities  as  a professional  who  teaches? 

1 1 . What  emphases  are  important  for  teaching  professionals? 

12.  What  is  your  product? 

13.  Who  is  your  customer?  External-  Internal- 

14.  What  was  different  about  the  definition  of  knowledge  presented  during  Orientation  than  what  most  would  expect  to  it 
to  be? 

15.  Learning  requires  some  major  shifts,  as  described  by  Senge.  What  are  they;  most  importantly,  what  do  they  reveal? 

16.  How  is  learning  constructed? 

17.  Where  does  knowledge  reside? 

18.  Generative,  in  the  context  we  are  in,  and  as  described  by  Senge,  means.... 

19.  How  is  "creative  tension"  important? 

20.  Describe  "transformative  learning"  and  "transformative  pedagogy".  What  result  does  transformative  pedagogy  have 
that  other  pedagogies  do  not? 

21.  What  meaning  does  Dale's  Cone  have  for  you? 

22.  What  meaning  does  Bloom's  Taxonomy  have  for  you? 

23.  What  is  the  difference  between  the  traditional  Bloom  and  the  Revised  Bloom's  Taxonomy? 
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Assessment  - Course  Analysis 


1.  List  and  describe  all  "teaching  styles"  you  know  about  or  use. 

2.  List  and  describe  all  student  "learning  styles"  you  know  about  or  use. 

3.  List  and  describe  all  "teaching  models"  you  know  about  or  use. 

4.  Describe  an  example  of  "active"  learning. 

5.  Describe  an  example  of  "passive"  learning. 

6.  Describe  what  "critical  thinking"  means  to  you  in  the  context  of  education  in  higher  education  and 
your  role  in  higher  education. 

7.  Describe  what  "teacher,"  "knowledge,"  "assessment,"  and  "learner"  centered  means  in  the  context 
of  higher  education. 

8.  What  does  "student  learning  objective"  mean?  Provide  an  example. 

9.  What  does  "student  learning  outcome"  mean?  Provide  an  example. 

10.  How  are  ABET  or  ABET  TAC/NAIT  standards  to  be  used? 

11.  Describe  "stages  of  learning." 

12.  What  meaning  does  “centered”  have  in  our  context? 
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Assessment  - Student  Learning  Objectives 

1.  Define  the  following  types  of  student  learning  statements: 

a.  Objective 

b.  Outcome 

c.  Behavioral  Objective  - how  does  this  one  differ  from  the  others? 

d.  Goal 

e.  Standard 

2.  List  a-e  above  in  priority  of  specificity,  broadest  statement  to  most  specific  statement. 

3.  Describe  each  type  of  objective  on  the  continuum  of  objectives,  global  educational  ->  instructional.  What 
purpose  does  each  one  serve?  Describe  the  nesting  priority,  i.e.  which  ones  are  nested  within  the  other(s)? 

4.  Are  objectives  the  “means  to  the  end”  or  the  “end”?  Why? 

5.  When  planning  student  learning,  what  model  is  accepted  as  the  best  practice? 

6.  What  is  essential  in  verb  choice  for  learning  statements? 

7.  Discuss  the  difference  between  “subject  matter  content”  and  “knowledge.” 

8.  Describe  the  difference  between  “content”  standards  and  “performance”  standards. 

9.  What  does  “KSA”  stand  for? 

10.  Describe  “mastery  learning  targets.” 

11.  What  are  “developmental  learning  targets”? 

12.  How  can  Bloom’s  Revised  Taxonomy  of  Learning  be  used  in  course  planning? 
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Test  Analysis 


Name: 


1.  In  the  context  of  item  analysis,  what  is  meant  by  the  term  item  difficulty! 

2.  In  the  context  of  item  analysis,  what  is  meant  by  the  term  item  discrimination! 

For  items  3 & 4 choose  the  BEST  answer. 

3.  An  item  in  a test  is  worth  3 points.  Ten  students  obtained  the  full  3 points,  seven  obtained  2 
points,  six  obtained  1 point,  and  two  students  obtained  zero  points.  What  is  the  item  difficulty  for 
this  item? 

A.  The  item  would  be  considered  very  easy  because  only  two  students  received  no  credit  for 
the  item. 

B.  The  item  would  be  considered  moderately  difficult  because  nine  students  (36%)  received 
either  two  points  or  one  point  for  the  item. 

C.  The  item  would  be  considered  very  difficult  because  only  ten  students  (40%)  received  full 
credit  for  the  item. 

D.  About  .40 

E.  About  .67 

F.  Can’t  tell  with  the  information  provided. 

4.  What  is  the  item  discrimination  for  the  item  presented  in  question  3 above? 

A.  The  item  would  be  judged  a highly  discriminating  item  because  ten  students  obtained  full 
credit  for  the  item  and  two  received  no  credit  for  the  item. 

B.  The  item  would  be  considered  a moderately  discriminating  item  because  the  students 
scores’  on  the  item  are  reasonably  distributed  (spread  out). 

C.  The  item  would  be  considered  a poorly  discriminating  item  because  a large  percentage  of 
the  students  (about  68%)  received  2 or  3 points  for  the  item. 

D.  About  .40 

E.  About  .67 

F.  Can’t  tell  with  the  information  provided. 

For  items  5 through  10  indicate  either  True  or  False. 

5.  If  an  item  is  very  difficult,  it  will  have  poor  item  discrimination. 

6.  A highly  discriminating  item  will  be  of  moderate  difficulty  (neither  very  difficult  nor  very 
easy). 

7.  A very  low  level  of  discrimination  for  an  item  indicates  that  the  item  probably  didn’t  work 
very  well  on  the  exam  and  that  the  item  should  be  considered  for  modification. 

8.  If  all  students  answer  an  item  correctly  (all  students  obtain  full  credit  for  the  item),  then  the 
item  is  too  easy  and,  therefore,  should  not  be  included  in  the  exam  because  it  doesn’t 
contribute  to  the  variation  of  students’  scores. 


More  items  on  back. 


9.  If,  as  a result  of  item  analysis,  an  item  is  judged  to  be  extremely  difficult,  then  either  the 
instructor  did  a poor  job  teaching  or  the  students  did  a poor  job  learning. 

10.  For  a given  exam,  the  sum  of  students’  scores  on  the  exam  is  mathematically  equal  to  the 
sum  of  items’  scores  on  the  exam. 
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Test  Analysis 


Name: 


1.  In  the  context  of  item  analysis,  what  is  meant  by  the  term  item  difficulty! 

The  proportion  of  points  awarded  from  an  item  to  the  number  of  points  that  could 
be  awarded  from  the  item. 

2.  In  the  context  of  item  analysis,  what  is  meant  by  the  term  item  discrimination! 

The  correlation,  across  students,  between  item  scores  and  total  test  scores. 

For  items  3 & 4 choose  the  BEST  answer. 


3.  An  item  in  a test  is  worth  3 points.  Ten  students  obtained  the  full  3 points,  seven  obtained  2 
points,  six  obtained  1 point,  and  two  students  obtained  zero  points.  What  is  the  item  difficulty  for 
this  item? 

A.  The  item  would  be  considered  very  easy  because  only  two  students  received  no  credit  for 
the  item. 

B.  The  item  would  be  considered  moderately  difficult  because  nine  students  (36%)  received 
either  two  points  or  one  point  for  the  item. 

C.  The  item  would  be  considered  very  difficult  because  only  ten  students  (40%)  received  full 
credit  for  the  item. 

D.  About  .40 
About  .67 

F.  Can’t  tell  with  the  information  provided. 


4.  What  is  the  item  discrimination  for  the  item  presented  in  question  3 above? 

A.  The  item  would  be  judged  a highly  discriminating  item  because  ten  students  obtained  full 
credit  for  the  item  and  two  received  no  credit  for  the  item. 

B.  The  item  would  be  considered  a moderately  discriminating  item  because  the  students 
scores’  on  the  item  are  reasonably  distributed  (spread  out). 

C.  The  item  would  be  considered  a poorly  discriminating  item  because  a large  percentage  of 
the  students  (about  68%)  received  2 or  3 points  for  the  item. 

D.  About  .40 
About  .67 

Can’t  tell  with  the  information  provided. 

For  items  5 through  10  indicate  either  True  or  False. 


5.  If  an  item  is  very  difficult,  it  will  have  poor  item  discrimination. 


False 


6.  A highly  discriminating  item  will  be  of  moderate  difficulty  (neither  very  difficult  nor  very 
easy). 


False 


More  items  on  back. 


7.  A very  low  level  of  discrimination  for  an  item  indicates  that  the  item  probably  didn’t  work 
very  well  on  the  exam  and  that  the  item  should  be  considered  for  modification. 

True 

8.  If  all  students  answer  an  item  correctly  (all  students  obtain  full  credit  for  the  item),  then  the 
item  is  too  easy,  and,  therefore,  should  not  be  included  in  the  exam  because  it  doesn’t 
contribute  to  the  variation  of  students’  scores. 

False 

9.  If,  as  a result  of  item  analysis,  an  item  is  judged  to  be  extremely  difficult,  then  either  the 
instructor  did  a poor  job  teaching  or  the  students  did  a poor  job  learning. 

False 

10.  For  a given  exam,  the  sum  of  students’  scores  on  the  exam  is  mathematically  equal  to  the 
sum  of  items’  scores  on  the  exam. 

True 
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Test  ] 

Rubric  (Scarborough,  2006) 

Test 

Characteristics 

Pts. 

Adequate  Quality  Level  - Not  Acceptable  (our  project) 

Pts 

Model  of  Excellence  - Required  Quality  Level  (our  project) 

Table  of 
Specifications 

1 

Many  content  areas  are  covered;  some  important  areas  are  not 

Knowledge  is  the  primary  level  covered  in  the  test. 

Many  items  are  directly  linked  to  Learning  Outcomes(Objectives) 
-Master  Targets,  but  not  for  entire  course  - Primary  and  highly 
prioritized  areas  covered 

Test  includes  only  THREE  item  types 

2 

All  or  almost  all  of  the  content  areas  are  sampled  - are  covered  by  test  items 
Test  should  include  knowledge,  comprehension,  and  application  levels 

Items  are  linked  directly  to  Learning  Outcomes  (Objectives.)-Master  Targets  for  entire  course 
Test  should  include  multiple  item  types,  5 types  (for  our  project) 

Overall 

Appearance 

1 

Test  may  have  a few  inconsistencies  in  layout  and  appearance;  it 
may  require  some,  but  not  major,  modification;  grammar, 
wording,  organization  is  appropriate  and  professional 

2 

Test  should  be  professionally  laid  out,  with  consistent  formatting  and  a generally  pleasing  appearance;  good  grammar,  wording, 
organization 

Overall 

Instructions  for 
the  Exam 

1 

Instructions  are  not  direct,  include  few  ambiguities,  are  too  long, 
or  require  students  to  assume  what  they  are  supposed  to  do,  how 
they  are  supposed  to  answer. 

2 

Instructions  should  be  direct,  clear,  unambiguous,  concise,  and  easy  to  follow 

Instructions  for 
Specific  Subsets 
of  Items 

1 

Instructions  are  not  direct,  include  few  ambiguities,  are  too  long, 
or  require  students  to  assume  what  to  do  and  how  to  answer. 

2 

Instructions  should  be  direct,  clear,  unambiguous,  concise,  with  appropriate  reading  level,  and  easy  to  follow 

Item  Quality 

1 

A few,  but  not  many,  items  are  less  well  written,  ambiguous, 
and/or  do  not  conform  to  item  writing  guidelines. 

2 

Clear,  direct,  well-written,  with  no  clues.  They  basically  conform  to  item  writing  guidelines. 

Fair  for  a 
diverse  range  of 
learners 

1 

Many  examinees  finish  early,  or  a few,  but  not  many,  examinees 
don’t  finish  in  the  allotted  time 

Most  of  the  language  is  at  an  appropriate  level,  but  some 
language  or  vocabulary  may  be  too  elementary  or  too  difficult 

Regarding  race,  ethnicity,  gender,  only  Excellent  is  acceptable 

2 

Appropriate  time  for  test  administration  (virtually  all  examinees  should  complete  the  exam  in  the  allotted  time) 
Appropriate  language/vocabulary  level  for  course  level 

Appropriate  race,  ethnic,  gender,  cultural  references  - should  be  race-  and  ethnic-  cultural-gender-balanced  (or  neutral) 

Item  Difficulty 

1 

A few  items  are  too  easy  or  too  difficult 

2 

No  items  are  too  difficult  or  too  easy 
(can  be  confirmed  with  post-testing  item  analysis). 

Item 

Discrimination 

1 

A few  items  have  low  (less  than  0.10)  or  negative  discrimination 

2 

No  items  have  discriminations  less  than  0.10  (can  be  confirmed  with  post-testing  item  analysis). 
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PERFORMANCE  TASK  AND  RUBRIC  DEVELOPMENT  ASSESSMENT 

Jule  Dee  Scarborough,  Ph.D. 


The  learning  and  professional  growth  on  the  Performance  Assessment  program  or  knowledge  component 
was  measured  by  the  professors’  performance  on  the  task  of  designing  and  developing  three  complex 
performance  tasks  and  three  corresponding  rubrics  for  scoring  task  achievement.  Using  the  Rubrics  below 
as  guiding  criteria,  they  each  designed  three  complex  performance  tasks  and  corresponding  rubrics.  These 
assessments  were  added  to  their  course  a new  assessment  strategy  and  assessment  procedures. 

It  is  important  to  note  that  one  performance  task/rubric  was  designed  to  correspond  with  the  midterm,  and 
another  to  correspond  with  the  final  exam  using  the  logic  that  objective  tests  usually  reflect  what  students 
know  or  know  about  rather  than  what  they  can  do.  Therefore,  we  used  an  unusual  scenario  where  the 
professors  “linked”  the  objective  midterm  exam  to  a midterm  performance  task/rubric  as  well  as  an 
objective  final  exam  to  a final  performance  task/rubric.  They  also  developed  a third  performance 
task/rubric  and  choose  how  and  when  to  use  it.  They  were  asked  to  “match”  where  they  thought  the  test 
items  and  performances  “overlapped”  and  measured  the  same  or  similar  content.  It  was  assumed  from 
studying  the  literature  that  performance  assessment  measures  different  aspects  of  learning,  sometimes 
deeper  levels  of  learning  through  use  of  knowledge  in  more  active  or  engaging  ways,  problems,  projects, 
etc.  But  performance  assessment  can  also  measure  some  of  the  same  aspects  of  learning  as  objective  tests. 
Also  some  of  the  professors  designed  their  tests  to  incorporate  some  level  of  performance  in  subjective  or 
problem  based  items.  In  examining  the  tests  and  analyzing  them,  the  objective  items  were  separated  from 
the  more  performance  based  items. 

Professors  were  provided  a presentation  about  Performance  Assessment.  Performance  Tasks  and  Rubrics 
were  discussed,  and  they  received  a portfolio  of  sample  tasks  and  rubrics.  They  were  given  books  on  the 
topic  as  part  of  their  new  library  on  teaching  and  learning.  Their  performance  tasks  and  rubrics  reflect  the 
ABET  or  NAIT  standards  with  corresponding  rubrics.  Perhaps  one  professor  had  used  simple  and  less 
formalized  rubrics  before,  but  none  of  the  professors  had  developed  or  used  formal,  written,  scenario- 
based  performance  tasks  with  corresponding  rubrics  before  this  initiative.  Thus,  there  were  no  previous 
instruments  to  view  from  the  baseline  semester.  Fall  2005,  and  compare  to  these.  Therefore,  we  judged 
them  based  upon  the  Rubrics  below. 

Performance  Task:  Design  and  develop  three  complex  performance  tasks  with  corresponding  rubrics. 
The  tasks  must  be  based  upon  the  ABET  outcomes  or  NAIT  standards  and  corresponding  student 
learning  outcomes  for  the  course;  they  must  also  reflect  real  world,  authentic  performances,  tasks,  or 
behaviors  in  the  appropriate  community  of  practice,  e.g.  industry.  The  performance  tasks  and  rubrics 
must  be  used  to  measure  student  learning  in  the  experimental  research  course.  Fall  2006.  See  the  Rubrics 
below  for  the  achievement  criteria  to  use  in  accomplishing  the  task. 

Performance:  The  professors  accomplished  the  performance  task  well.  The  process  involved  drafting 
initial  and  authentic  real  world  scenarios  with  embedded  task  clusters  and  a corresponding  rubric 
instrument  for  each  task  (3).  The  program  leader  provided  feedback  one-on-one  as  the  performance  tasks 
were  developed.  The  professors  shared  their  drafts  with  each  other  and  benefited  from  the  group  critique 
process.  The  group  process  worked  especially  well.  The  tasks  and  rubrics  were  finalized;  the  program 
leader  approved  them;  and  then,  each  professor  used  the  tasks  and  corresponding  rubrics  successfully  with 
students  during  the  2006  experimental  research  semester.  After  the  semester  was  completed,  the 
professors  copied  all  rubrics  returned  to  each  student  in  their  classes  for  all  three  performance  tasks.  The 
program  leader  reviewed  the  scored/with  comments  rubrics  that  each  student  received  back  from  the 
professors.  Thus,  the  use  of  the  rubrics  was  also  reviewed.  Finally,  the  professors  completed  a 
feedback/evaluation  form  about  the  use  of  performance  assessment  for  the  first  time.  As  with  test  analysis 
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and  development,  the  feedback  from  the  professors  on  the  value  of  learning  to  design,  develop,  and  use 
performance  tasks/rubrics  was  extremely  positive. 

The  following  rubrics  were  used  to  guide  the  professors  in  the  development  of  the  three  performance  tasks 
and  corresponding  rubrics  for  each  task. 

Also,  the  feedback  and  evaluation  questionnaire  and  professor  responses  are  provided  below,  following 
the  rubrics.  The  faculty  members  truly  felt  that  expanding  their  assessments  to  include  performance  tasks 
with  rubrics  was  extremely  positive.  They  all  indicated  that  they  will  continue  to  use  performance 
assessment,  tasks  and  rubrics,  and  also  expand  the  use  of  performance  assessment  to  other  courses. 

Responses  to  the  feedback  and  evaluation  related  to  Performance  Assessment  follows  the  rubrics  below. 
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Rubric  for  Assessing  the  Quality  of  a Performance  Task 

Key  Components  - Properly  Designed  Performance  Tasks  must 

I.  Be  based  on  content  standards  established  by  ABET  or  NAIT 

II.  Describe  a “real-life”  scenario;  are  real  world,  authentic  tasks;  require  active  performances 

III.  Involve  students  in  complex  reasoning  - critical  thinking  at  upper  levels  of  Bloom’s  Cognitive  Dimension 

IV.  Require  students  to  collect  and  process  information,  using  it  for  an  authentic  purpose 

V.  Incorporate  “habits  of  mind” 

VI.  Require  student  collaboration  and  cooperation;  incorporates  “individual”  and  “group”  learning  and  performance 
accountability 

VII.  Result  in  a tangible  product  and/or  communication  activity 

For  each  component,  there  are  descriptors  reflecting  levels  of  achievement  possible: 

I.  The  Performance  Task  is  based  on  the  ABET  or  NAIT  standards 

a.  The  Performance  Task  is  directly  related  to  the  ABET  or  NAIT  standards. 

b.  Learning  standards  are  apparent,  but  the  relation  to  the  task  and/or  national  standards  is  sketchy  or  not  apparent. 

c.  The  Performance  Task  does  not  appear  to  be  based  on  the  standards/outcomes,  course  or  national. 

II.  The  Performance  Task  describes  a “real-life”  scenario  that  is  authentic  and  requires  active  performance. 

a.  The  scenario  described  in  the  task  accurately  mirrors  an  activity  in  the  community  of  practice  outside  the  classroom. 

b.  The  scenario  described  in  the  task  simulates  an  activity  in  the  community  of  practice  outside  the  classroom. 

c.  The  scenario  described  in  the  task  contains  some  aspects  of  activity  outside  the  classroom  but  is  largely  contrived. 

d.  The  scenario  described  in  the  task  is  an  academic  exercise  that  usually  takes  place  only  in  the  context  of  an  academic  setting. 

III.  The  Performance  Task  involves  students  in  complex  reasoning-critical  thinking  processes  at  upper  levels  of  Bloom’s 
Cognitive  Dimension. 

a.  The  task  requires  students  to  utilize  complex  reasoning  - critical  thinking  skills,  such  as  induction/deduction,  diagnosis, 
abstracting,  experimental  inquiry,  problem  solving;  evaluation,  creation,  synthesis,  etc. 

b.  The  task  requires  students  to  utilize  complex  reasoning  components,  such  as  comparing,  classifying,  decision  making,  or 
investigation. 

c.  The  task  requires  students  only  to  recall  facts. 

IV.  The  Performance  Task  requires  students  to  collect  and  process  information,  using  it  for  an  authentic  purpose. 

a.  The  task  incorporates  a variety  of  information  gathering  techniques  and  information  resources.  Students  are  required  to 
interpret  and  synthesize  information  and  accurately  assess  the  value  of  information  gathered.  They  are  required  to  collect  the 
right  information  for  an  authentic  purpose,  e.g.  solve  a problem,  apply  or  use  in  a complex  project,  etc. 

b.  The  task  requires  students  to  gather  and  synthesize  information,  but  the  value  of  the  information  gathered  is  not  assessed. 
Information  may  not  be  used  for  a purpose. 

c.  The  task  requires  the  students  to  gather  information,  but  not  to  interpret  it. 

d.  The  task  requires  no  gathering  or  processing  of  information. 

V.  The  Performance  Task  incorporates  “Habits  of  Mind.” 

a.  The  task  requires  students  to  make  effective  plans,  use  necessary  resources,  evaluate  effectiveness  of  their  own  actions,  seek 
accuracy,  and  engage  in  activities  when  answers  or  solutions  are  not  immediately  apparent. 

b.  The  task  only  requires  students  to  effectively  plan  or  use  resources. 

c.  The  task  does  not  require  students  to  engage  in  self-regulation,  critical,  or  creative  thinking. 

VI.  The  Performance  Task  requires  student  collaboration  and  cooperation;  incorporates  “individual”  and  “group”  learning 
and  performance  accountability. 

a.  The  task  requires  students  to  use  interpersonal  skills,  work  toward  the  achievement  of  team  goals,  and  perform  a variety  of 
roles  within  the  team.  There  is  a formal  team  structure  and  process. 

b.  The  task  requires  students  to  work  together  in  teams  but  there  are  no  measures  described  that  ensure  collaboration  or 
cooperation  among  team  members. 

c.  The  task  is  completed  largely  by  students  on  an  individual  basis  rather  than  in  student  teams. 

VII.  The  Performance  Task  results  in  a tangible  product  and/or  communication  activity. 

a.  The  task  result  is  a tangible  product  or  communication  activity  comparable  to  that  commonly  produced  in  business  or  industry 
community  of  practice. 

b.  The  task  results  in  a product  that  is  similar  to  those  completed  in  business  or  industry  community  of  practice,  but  lacks  several 
components  that  make  the  product  realistic. 

c.  The  task  does  not  result  in  a product  or  communication  activity  relevant  to  a business  or  industry  community  of  practice. 
(Scarborough,  2006  [Based  upon  White  & Scarborough,  2004]) 
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Rubric  for  Assessing  the  Quality  of  a Rubric 


Properly  Designed  Rubrics  Must 

I.  Contain  a set  of  key  components/standards  to  be  assessed  that  reflect  the  student  learning  outcomes 
for  the  course,  which  are  directly  linked  to  the  national  outcomes. 

II.  Include  descriptors  for  each  component/standard  that  are  measurable. 

III.  Have  descriptors-criteria  that  are  indicative  of  observable  student  performances  or  behaviors. 

IV.  Incorporate  a clear  and  well-defined  scoring  system 

V.  (Optional)  Include  appropriate  weights  for  each  component  and  descriptor 

For  each  component,  there  are  descriptors  reflecting  levels  of  achievement  possible: 

I.  The  rubric  contains  a set  of  key  components  (standards)  to  be  assessed. 

a.  A complete  list  of  key  components-standards  is  provided  for  the  performance  task, 
including  the  embedded  subtasks,  if  a cluster.  The  task(s)  are  directly  connected  to 
student  learning  outcomes  for  course  and  the  national  outcomes. 

b.  Key  components/standards  listed  are  not  exhaustive  for  the  performance  task  and/or 

subtasks  embedded  are  not  clear  enough  for  student  response  or  action;  components  or  standards  are  not  clearly 
connected  to  student  learning  outcomes  for  course. 

c.  Not  all  key  components/standards  describe  student  outcomes;  some  are  not  directly  linked  to  national 
outcomes. 

d.  No  key  components  are  listed. 

II.  The  rubric  includes  a set  of  descriptors-criteria  for  each  key  component  or  standard. 

a.  Descriptors-criteria  for  each  component  or  standard  are  arranged  in  a clear  hierarchy  from  non-achievement  to  full- 
achievement. 

b.  Descriptors-criteria  are  present  for  each  component/standard,  but  obvious  levels  in  some  are  missing. 

c.  Each  component  does  not  have  an  associated  set  of  descriptors-criteria. 

III.  The  rubric  descriptors/criteria  are  clear  and  contain  observable  or  measurable  student 
performances  or  behaviors. 

a.  All  descriptors-criteria  clearly  delineate  levels  of  observable  student  performances  or  behaviors. 

b.  Most  descriptors-criteria  clearly  delineate  levels  of  observable  student  performances  or  behaviors. 

c.  Only  a few  descriptors-criteria  clearly  define  levels  of  observable  student  performances  or  behaviors. 

d.  Descriptors-criteria  do  not  describe  observable  student  performances  or  behaviors. 

IV.  Incorporate  a clear  and  well-defined  scoring  system 

a.  There  is  a well  defined  and  clear  system  for  scoring  each  component-standard  and  its  descriptors-  criteria.  Points  or 
percentages  are  assigned  appropriate  to  instructional  and  performance  values. 

b.  The  scoring  system  lacks  definition,  clarity,  and  although  there  is  a scoring  system,  some  aspects  are 
ambiguous,  subjective  or  unclear. 

c.  There  is  no  scoring  system. 

V.  Optional:  Appropriate  weights  are  assigned  to  components  and  descriptors. 

a.  Component-standards  and  descriptors-criteria  are  each  properly  weighted  according  to 

instructional  emphasis  and  performance  values. 

b.  Weights  are  assigned,  but  point  values  do  not  reflect  proper  instructional  emphasis  and  performance  values  in  all 
cases. 

c.  Weights  are  assigned  to  some  performance  standards  and  descriptors,  but  not  others. 


(Scarborough,  2006  [Based  upon  White  & Scarborough,  2004]) 


Research  Designs  and  Inferential  Statistical  Procedures 


Name: Date: 

For  items  1-6,  select  the  letter  from  the  lists  below  (A  - M)  that  answers  the  question  or  is  the  best 
correspondence  to  the  statement. 

Experimental  Designs: 

A.  Design  1.  The  One-Shot  Case  Study:  X O 

B.  Design  2.  The  One-Group  Pretest-Posttest  Design:  0X0 

C.  Design  3.  Static-Group  Comparison:  X 


D.  Design  4.  Pretest-Posttest  Control  Group  Design: 


R O X 

R O 


E.  Design  5.  Solomon  Eour-Group  Design: 


R O X 

R O 

R X 

R 


E.  Design  6.  Posttest-Only  Control  Group  Design:  R 

R 


X 


Threats  to  Internal  Validity: 

G.  History 

H.  Maturation 

I.  Testing 

J.  Instrumentation 

K.  Regression 
E.  Selection 
M.  Mortality 


1.  These  three  experimental  designs  are  considered pre-experimental  designs: 

, , and . 

2.  These  three  experimental  designs  are  considered  true  experimental  designs: 

, , and . 
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oo  oo  oooo  oo 


3.  This  threat  to  internal  validity  refers  to  events  that  occur  between  the  pretest  and  posttest, 

in  addition  to  the  treatment,  that  could  affect  the  posttest  scores.  . 

4.  This  threat  to  internal  validity  refers  to  the  test  or  testing  process  being  different  at 

posttest  than  it  was  at  pretest.  . 

5.  Which  experimental  design  is  the  most  used  design  and  sometimes  called  the  classical 

experimental  design?  . 

6.  This  is  a design  that  does  not  control  for  the  effects  of  history  and  maturation.  . 

(More  than  one  answer  is  possible.) 


7.  Posttest  scores  in  an  experimental  design  involving  experimental  and  control  groups  with 

random  assignment  of  subjects  to  groups  assures  that  the  threats  to  internal  validity  are 
not  only  controlled,  but  eliminated.  True  False 

8.  Retaining  the  null  hypothesis  is  consistent  with  inferring  that  the  treatment,  X,  had  no 
effect  on  posttest  scores.  True  False 

9.  Inferential  statistical  procedures  allow  us  to  conclude  that  either  the  treatment  was 
effective  or  it  was  not.  True  False 

10.  The  basic  experimental  inference  we  wish  to  make  after  we’ve  concluded  our  research  is  that 

A.  the  posttest  mean  is  higher  than  the  pretest  mean. 

B.  the  null  hypothesis  is  true. 

C.  the  treatment , X,  caused  or  was  directly  related  to  the  observations  made. 

D.  the  treatment,  X,  was  one  possible  cause  for  the  difference  between  the  pretest  and 
posttest  means. 
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Points  possible  =14 


Research  Designs  and  Inferential  Statistical  Procedures 

Name: Date: 

For  items  1-6,  select  the  letter  from  the  lists  below  (A  - M)  that  answers  the  question  or  is  the  best 
correspondence  to  the  statement. 

Experimental  Designs: 

A.  Design  1.  The  One-Shot  Case  Study:  X O 

B.  Design  2.  The  One-Group  Pretest-Posttest  Design:  0X0 

C.  Design  3.  Static-Group  Comparison:  X 


D.  Design  4.  Pretest-Posttest  Control  Group  Design: 


R O X 

R O 


E.  Design  5.  Solomon  Eour-Group  Design: 


R O X 

R O 

R X 

R 


E.  Design  6.  Posttest-Only  Control  Group  Design:  R 

R 


X 


Threats  to  Internal  Validity: 

G.  History 

H.  Maturation 

I.  Testing 

J.  Instrumentation 

K.  Regression 
E.  Selection 
M.  Mortality 


1.  These  three  experimental  designs  are  considered pre-experimental  designs: 

A B and  C 

2.  These  three  experimental  designs  are  considered  true  experimental  designs: 

D E and  F 
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oo  oo  oooo  oo 


3.  This  threat  to  internal  validity  refers  to  events  that  occur  between  the  pretest  and  posttest, 
in  addition  to  the  treatment,  that  could  affect  the  posttest  scores.  G . 


4.  This  threat  to  internal  validity  refers  to  the  test  or  testing  process  being  different  at 
posttest  than  it  was  at  pretest.  J . 

5.  Which  experimental  design  is  the  most  used  design  and  sometimes  called  the  classical 
experimental  design?  D . 

6.  This  is  a design  that  does  not  control  for  the  effects  of  history  and  maturation.  A or  B 

(More  than  one  answer  is  possible.) 


7.  Posttest  scores  in  an  experimental  design  involving  experimental  and  control  groups  with 
random  assignment  of  subjects  to  groups  assures  that  the  threats  to  internal  validity  are 
not  only  controlled,  but  eliminated.  TrueC^Falsey 


8.  Retaining  the  null  hypothesis  is  consistent  with  inferring  that  the  treatment,  X,  had  no 
effect  on  posttest  scores.  (^/True^  False 


9.  Inferential  statistical  procedures  allow  us  to  conclude  that  either  the  treatment  was 
effective  or  it  was  not.  True  C False 


10.  The  basic  experimental  inference  we  wish  to  make  after  we’ve  concluded  our  research  is  that 

A.  the  posttest  mean  is  higher  than  the  pretest  mean. 

B.  the  null  hypothesis  is  true. 

(O  the  treatment , X,  caused  or  was  directly  related  to  the  observations  made. 

D.  the  treatment,  X,  was  one  possible  cause  for  the  difference  between  the  pretest  and 
posttest  means. 
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(COPIED  FROM  B.O) 


PORTFOLIO  SUMMARY  COMMENTS 
Jule  Dee  Scarborough,  Ph.D. 

(See  Tables  I and  2 below) 

The  Portfolio  Assessment  Chart  below  reveals  that  all  professors  except  one  completed 
all  aspects  of  the  program  and  research  semester  successfully.  That  means  that  each  of 
those  seven  professors  completed  the  faculty  development  program  of  learning  with 
significant  gain  in  learning.  Seven  professors  completed  all  teaching  and  learning 
products  during  the  faculty  development  program;  and  all  seven  professors  fully 
participated  in  the  research  semester,  executing  experimental  research  in  the  classroom 
with  their  students.  All  seven  professors  prepared  a research  manuscript  and  submitted  it 
for  publication.  However,  one  of  the  seven  professors  did  not  complete  some  activities  as 
planned;  he/she  did  not  diagnostically  analyze  the  final  examination  and  did  not  use  the 
second  and  third  performance  assessments  as  planned.  That  individual  did  complete  the 
research  as  planned  but  did  not  implement  the  full  range  of  changes  prepared  and  planned 
for  the  2006  course.  This  culminating  assessment,  college  portfolio,  provides  evidence 
that  the  program  was  very  successful,  resulting  in  significant  change  and  a new  range  of 
teaching  and  learning  activities  for  each  professor.  The  portfolio  also  reflects  each 
professor’s  preparation  for  the  research  semester,  itemizing  the  products  developed  and 
used  during  the  2006  experimental  research  semester.  Generally,  the  portfolio  chart 
reveals  the  results  of  the  faculty  development  program  and  research  semester  and 
documents  the  professors’  learning  and  progress  toward  new  teaching  and  learning 
strategies,  as  well  as  that  toward  the  Scholarship  of  Teaching. 

Teaching  Portfolio  Assessment  Forms  and  Check-Off  Forms  Below 
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Teaching  Portfolio  Assessment  Chart  Form,  DATE  - CITL  Faculty  Development  Program 


Portfolio  Product  (Artifact)  Content 
(See  Sections  of  information  following  this  summary) 

Faculty 

Member 

Faculty 

Member 

Self  Assessment  Baseline: 

1 . Student  Questionnaires  (f05  & f06) 

1 .  Professor  completion(s)  of  Student  Questionnaire  (f05  &f06) 

2.  Professor  completions  of  Self  Competency  Questionnaire 

(Feb.06,  May06,  Dec.06) 

3.  Program  Components  Assessments  (8) 

4.  Standard  Departmental  Course  Evaluations  (f05  & f06) 

4. Student  Grades  & End  of  Semester  Grades  (f05  & f06) 

5.  Course  Analysis: 

5al._Course  Outline,  Embedded  Gen  Ed,  Content  Priorities 
5a2._Course  Content  Analysis  by  TM,TS,  LS,  Bl,  Dale,  etc. 

5b.  _ Instr.  Design  GAPS  Analysis  on-  TM,  TS,  LS,  B,  D 
5c.  _ Instructional  GAPS  Summary 
5d.  _ ABET/TAC/NAIT  SLO  by  Bloom’s  Analysis 
5e.  _ Course  Content  Schedule 

5f._  Teaching  Models+Cooperative  Learning+Study  Chart+TM  graphic 
5g._  Course  Calendar  by  TM,  TS,  LS,  B,  D 

***Student  Learning  Styles  Inventory:***  (NOT  REQUIRED) 
Kolb  (Extra  professional  effort  on  part  of  professors) 

Felder  (Extra  professional  effort  on  part  of  professor) 

Multifaceted  Assessment  System: 

5h. Multifacted  Assessment  Plan  Graphic,  showing  course  assessments 

5i. Test  and  Test  Items  by  SLO  Chart 

5j. Assessment  Analysis  by  Bloom  (Chart) 

6.  Traditional  Objective  Tests:: 

Test  Analysis  (Midterm  and  Final  Exam) 
Table  of  Specifications  (not  included) 
Test  Item  Bank  (not  included) 

7.  New  Midterm  Exam 

7.  New  Final  Exam 

8.  Diagnostic  Write  Ups  (MT  & F) 

Performance  Assessment  & Rubrics: 

7.  3 Complex  Performance  Assessments  with  multiple  tasks 

embedded 

7.  3 Rubrics,  one  to  score  each  Performance  Assessment 

( And  to  be  used  with  students  to  establish  standards  up  front) 

8.  Diagnostic  Write  Ups  (PA  1.2.3) 

* Copies  of  Students  Rubrics  (Hardcopies  on  file) 

7.  Electronic  copies  of  tests  and  PAs  & Rubrics 

other  Assessments  of  Individual  Choice:  Yes  for  All 

9.  Student  Centered  Course  Syllabus: 

All  new  components  and  check  off  list 

10.  Professors’  Research: 

Completed  Data  Forms  (including  data  on  MT,  F,  PA1,2,3) 

Research  Results  Reports 

12. Teaching  Portfolio  Assessment  Questionnaire 

13  Teaching  Models  Self  Assessment 

14  Teaching  Styles  Self  Assessment 

15  Student  Learning  Style  Opportunities  Assessment 

16  Outcomes  Achieved  as  Planned  by  Bloom  & Dale  Assessment 

17.  Manuscript  to  be  submitted: 

Draft 

Final  Version  to  be  submitted  to  journal  tMav.  2007) 

2 


Teaching  Portfolio  Assessment  Process  - May  25,  2006 
CITL  Professional  Development  Program 
Jule  Scarborough,  2006 


Portfolio  Product  (Artifact)  Content 

Self  Assessment  Baseline: 

Student  Questionnaire 

Professor  completion  of  Student  Questionnaire 
Professor  Self  Competency  Questionnaire 

Description  of  my  Knowledge, 
Skills,  Abilities  on  Feb.  2,  2006 

Description  of  my  Knowledge, 
Skills,  Abilities  on  May  25*'', 
2006 

Course  Analysis: 

GAPS  Analysis  on  TM.  TS.  LS.  B.  D 
ABET/TAC/NAIT  SLQ  by  Bloom’s  Analysis 
Course  Calendar  by  TM.  TS.  LS.  B.  D 
Teaching  Models  + Cooperatiye  Learning  + 
Mapping  Study  Chart 

Student  Centered  Course  Syllabus: 

All  new  components  and  check  off  list 

Portfolio  Product  (Artifact)  Content 

Multifaceted  Assessment  System: 

Course  Assessment  Plan  Chart  showing  course 
assessments 

Assessment  Analysis  by  Bloom  (Chart) 

Traditional  Objective  Tests: 

Test  Analysis 

Table  of  Specifications 

Test  Item  Bank 

New  Midterm  Exam 

New  Einal  Exam 

Test  Items  by  SLQ  Chart 

Performance  Assessment  & Rubrics: 

3 Complex  Performance  Assessments  with 
multiple  tasks  embedded 

3 Rubrics,  one  to  score  each  Performance 
Assessment  (and  to  be  used  with  students  to  establish 
standards  up  front) 

Other  Assessments  of  Individual  Choice: 

List  and  Describe  Here: 

1.  3. 

2.  4. 
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CEET  Initiative  on  Teaching  and  Learning 
Orientation  Feedback 


Feedback  inquiries  will  change  across  the  different  types  of  modules  with  some  constant  items. 
However,  both  content  assessment  and  quality  feedback  instruments  will  be  used  throughout  the 
program.  Below  is  a summary  of  faculty  feedback  offered  by  participants  for  the  one-day 
orientation  session. 

1.  Was  today  worth  your  time  - worthwhile? 

Yes  Not  really 

Why? 

2.  Would  you  recommend  today's  content  for  other  faculty  members? 

Yes  Not  really 

Why? 

3.  Will  the  survey  data  be  a useful  guide  for  instructional  decision-making? 

Yes  Not  really 

Why? 

4.  Would  you  recommend  that  other  faculty  members  get  the  opportunity  to  use  the  student 
survey? 

Yes  Not  really 

Why? 

5.  Was  the  "process"  used  today  effective? 

Yes  Not  really 

Why? 

6.  Strengths  of  today's  program. 

7.  Areas  to  improve  today's  program. 

8.  General  comments: 
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CEET  Initiative  on  Teaching  and  Learning 
Analysis  Eeedback-  Day  2 

1.  Was  today  worth  your  time  - worthwhile? 

Yes  Not  really 

Why? 

2.  Would  you  recommend  today’s  content  for  other  faculty  members? 

Yes  Not  really 

Why? 

3.  Would  you  recommend  that  other  faculty  members  get  the  opportunity  to  engage  in  this  preliminary 
analysis? 

Yes 

4.  Was  the  “process”  used  today  effective? 

Yes  Not  really 

Why? 

5.  Strengths  of  today’s  program. 

6.  Areas  to  improve  today’s  program. 

7.  General  Comments: 
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CEET  Initiative  on  Teaching  and  Learning 
Student  Learning  Outcomes  - Days  4 & 5,  March  2 & 23 


1.  Were  these  two  days  worth  your  time  - worthwhile? 

Yes  Not  really 

Why? 

2.  Would  you  recommend  the  program  content  on  Student  Learning  Outcomes  for  other  faculty  members? 

Yes  Not  really 

Why? 

3.  Did  the  presentation  and  handouts  provide  appropriate  information  to  guide  you  in  the  development  of  student 
learning  outcomes  without  overwhelming  you  with  too  much  material? 

Yes  No,  not  really 

Why? 


4.  Did  the  “student  learning  outcome”  sessions  help  you  to  specify  the  knowledge,  skills,  and  ability  course  content 
more  clearly  and  to  identify  priorities  more  logically? 

Yes  No,  not  really  - I had  already  done  a pretty  good  job  of  content  identification  and  prioritization 

Why? 

5.  Did  the  “student  learning  outcome”  session  help  you  to  develop  new  or  enhanced  student  learning  objectives? 

Yes,  they  are  more...  (select  all  that  apply) 

a.  intentional  in  content  and  result  or  outcome 

b.  results  oriented  - outcomes  oriented  in  that  they  clearly  state  what  students  are  to  know  about,  know,  or  be  able  to  do 

c.  specific  in  what  knowledge,  skill,  ability  is  to  be  learned  or  extended  by  the  student 

d.  measurable 

e.  observable 

f.  appropriately  stated  - using  more  definite  verbs  and  nouns;  they  explain  the  purpose,  provide  context,  situation, 
conditions,  etc. 

f.  all  of  the  above 

Not  really 

Why? 

6.  In  completing  the  calendar  for  the  formal  scheduling  of  course  content,  there  is  greater  potential  to 

(select  all  that  apply) 

a.  enhance  or  improve  the  course  focus 

b.  provide  a better  format  for  on-going  critical  analysis  of  the  course  content  as  updates  or  changes  are  needed 
b.  enhance  or  improve  the  course  content  delivery 

d.  help  me  better  visualize  my  course  and  how  to  continuously  update,  improve  or  enhance  it  to  continuously  increase 
student  learning 

e.  provide  the  students  with  a clearer  picture  of  the  course  and  what  they  are  to  learn 

f.  help  me  and  my  students  to  stay  on  “course” 

g.  all  of  the  above 

Not  really 
Why? 
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6.  Now  that  you  have  written  student  learning  outcomes,  do  you  feel  more  able  to  prepare  learning  activities  or 
experiences? 

Yes  No,  not  really 

Why? 

7.  Would  you  recommend  that  other  faculty  members  get  the  opportunity  to  engage  in  revising  their  course  Student 
Learning  Outcomes  - content  identification  and  student  learning  outcomes? 

Yes  Not  really  - too  soon  to  use  until  we  finish  our  program  and  further  explore  its  meaning 

Why? 

8.  Was  the  “process”  used  during  these  days  effective? 

Yes  Not  really 

Why? 

9.  Strengths  of  this  aspect  of  the  program. 

10.  Areas  to  improve  in  this  aspect  of  the  program. 

11.  General  Comments: 
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CEET  Initiative  on  Teaching  and  Learning 
Gap  Analysis  Summary  and  Test  Analysis  Eeedback-  Day  3 - Feb.  16, 2006 

1.  Was  today  worth  your  time  - worthwhile? 

Yes  Not  really 

Why? 

2.  Would  you  recommend  today’s  content  for  other  faculty  members? 

Yes  Not  really 

Why? 

3.  Did  the  GAPs  Analysis  & Summary  help  you  to  see  possibilities  for  extending  or  enhancing:  course  content 
and  teaching/learning  strategies;  also,  did  it  make  you  aware  of  models  and  techniques  that  you  could  consider 
using  to  build  or  extend  student  learning  experiences  to  higher  levels  of  learning? 

Yes  Not  really 

Why? 

4.  Would  you  recommend  that  other  faculty  members  get  the  opportunity  to  engage  in  the  preliminary  aspect  of 
test  analysis? 

Yes  Not  really  - too  soon  to  use  until  we  finish  our  program  and  further  explore  its  meaning 

Why? 

5.  Was  the  “process”  used  today  effective? 

Yes  Not  really 

Why? 


6.  Strengths  of  today’s  program. 


7.  Areas  to  improve  today’s  program. 


8.  General  Comments: 


1 


CEET  Initiative  on  Teaching  and  Learning 
Item  Writing  and  Test  Development  - Days  6,  7,  8, 9 - March  30,  April  6,  20,  27 

1.  Were  these  four  days  worth  your  time  - worthwhile? 

Yes  No,  not  really 

Why? 

2.  Did  the  presentation  and  handouts  provide  appropriate  information  to  guide  you  in  item  writing  and  test 
development  without  overwhelming  you  with  too  much  material? 

Yes  No,  not  really 

Why? 

3.  Were  you  able,  during  the  four  sessions,  to  write  many  appropriate  and  valid  test  items  and  to  assemble  what 
you  would  consider  good  tests  or  tests  more  fully  developed  than  the  ones  you  were  using  before  now? 

Yes  No,  not  really 

Why? 

4.  As  a result  of  these  four  sessions  do  you  now  feel  you  have  greater  ability  and  confidence  in  writing  items  and 
developing  tests? 

Yes  No,  not  really 

Why? 

5.  Would  you  recommend  that  other  faculty  members  have  the  opportunity  to  learn  more  about  writing  items  and 
developing  tests  through  workshops  similar  to  these? 

Yes  No,  not  really  - too  soon  to  use  until  we  finish  our  program  and  further  explore  its  meaning 

Why? 

6.  Was  the  “process”  used  during  these  days  effective? 

Yes  No,  not  really 

Why? 

7.  Strengths  of  this  aspect  of  the  program. 

8.  Areas  to  improve  in  this  aspect  of  the  program. 

9.  General  Comments: 
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CEET  Initiative  on  Teaching  and  Learning  - Performance  Assessment  Feedback,  Jan.  2007 


1.  Was  the  time  spent  developing  performance  tasks  worth  your  time  --  worthwhile? 

Yes  Not  really 

Why? 

2.  Was  the  time  spent  developing  rubrics  for  scoring  the  performance  tasks  worth  your  time  -- 
worthwhile? 

Yes  Not  really 

Why? 

3.  Would  you  recommend  the  performance  task  program  content  for  other  faculty  members? 

Yes  Not  really 

Why? 

4.  Would  you  recommend  the  rubric  program  content  for  other  faculty  members? 

Yes  Not  really 

Why? 

5.  Were  the  performance  tasks  a beneficial  addition  to  the  student  assessment  plan  for  your  course? 

Yes  Not  really 

Why? 

6.  Were  the  rubrics  beneficial  for  scoring  the  performance  tasks? 

Yes  Not  really 

Why? 

7.  Were  the  performance  tasks  an  effective  tool  for  enhancing  student  learning? 

Yes  Not  really 

Why? 

8.  Were  the  performance  tasks  an  effective  tool  for  measuring  student  learning? 

Yes  Not  really 

Why? 
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9.  Were  the  rubrics  an  effective  tool  for  scoring  the  outcomes  of  student  performances  on  the  tasks? 


Yes  Not  really 

Why? 

10.  Were  the  rubrics  effective  in  helping  students  to  understand  more  about  what  you  expected  them  to 
do  by  revealing  the  standards  and  scoring  mechanism  with  them  up  front? 

Yes  Not  really 

Why? 

11.  Do  you  feel  that  more  formalized  performance  tasks  and  rubrics  improved  the  opportunity  for 
students  to  provide  evidence  of  learning? 

Yes  Not  really 

Why? 

12.  Would  you  recommend  that  other  faculty  members  get  the  opportunity  to  learn  about,  develop,  and 
use  performance  tasks  and  rubrics  as  student  assessment  tools? 

Yes  Not  really  - 

Why? 

13.  Was  the  performance/rubric  development  process  used  with  this  group  - ‘‘'‘developing  while  learning; 
presentation;  examples;  and,  one-on-one  feedback  - effective? 

Yes  Not  really 

Why? 

14.  Will  you  continue  to  use  performance  tasks  and  rubrics  in  this  and/or  other  classes? 

Yes  Not  really 

Why? 

14.  Strengths  of  the  performance  task/rubric  program  component. 


15.  Areas  to  improve  in  the  performance  task/rubric  program  component. 


16.  General  comments: 
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CEET  Initiative  on  Teaching  and  Learning 
Program  Eeedback-  Einal  Eeedback  -December  15,  2006 

1.  Looking  back,  and  after  the  research  semester,  do  you  feel  that  the  faculty  developm 
your  time  - worthwhile? 

Test  Analysis  and  Development  Review 
Performance  Assessment  and  Rubies 
Analyzing  all  your  assessments  by  Bloom 

Consideration  of  “broader”  assessments  and  mapping  your  assessments 
Teaching  Models,  including  Cooperative  Learning  and  Mapping  Analysis 
The  review  of  components  for  a more  revealing  syllabus  for  the  students 
The  review  and  consideration  of  Multicultural  aspects  of  courses. 

The  review  and  consideration  of  grading. 

The  educational  Research  Session 


Why  or  Why  not  to  any  of  the  above  statements? 

2.  Would  you  recommend  the  overall  program  content  as  you  experienced  it  for  other  faculty  members? 

Yes  Not  really 

Why? 

3.  Was  the  Research  Semester,  performing  experimental  classroom  research  with  students,  worth  your  time  and 
“worthwhile”? 


sessions  were  worth 

Yes 

Not  really 

Yes 

Not  really 

Yes 

Not  really 

Yes 

Not  really 

Yes 

Not  really 

Yes 

Not  really 

Yes 

Not  really 

Yes 

Not  really 

Yes 

Not  really 

Not  really 

Yes  Not  really 

Why  or  Why  not  to  any  of  the  above  statements? 

4.  Looking  back  at  the  semester,  was  the  re-development  of  your  course,  or  the  changes  to  your  course,  worthwhile 
when  used  with  students? 


Yes,  definitely  Yes,  most  of  them  Yes,  the  majority  of  them  Yes,  some  of  them 

Not  as  many  of  the  changes  were  as  valuable  as  I had  hoped  No,  very  few  of  them  were  successful  changes 

Why  or  Why  not? 

5.  Were  the  teaching  and  learning  materials  developed  during  the  faculty  development  sessions  worthwhile  when 
used  with  students? 

Yes,  definitely  Yes,  most  of  them  Yes,  the  majority  of  them  Yes,  some  of  them 

Not  as  many  of  the  materials  were  as  valuable  as  I had  hoped  No,  very  few  of  the  materials  were  successful 

Why  or  Why  not? 

6.  Lrom  the  perspective  of  the  entire  program,  faculty  development,  the  development  of  course  and  classroom 
materials,  and  the  educational  research  semester,  was  it  as  a “whole”  worth  your  time  - worthwhile? 

Yes  Not  really 

Why  or  Why  not? 
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7.  Would  you  recommend  that  other  faculty  members  get  the  opportunity  to  participate  in  this  program  with 
content  modifications? 

Yes  Not  really  - too  soon  to  determine 

What  changes  would  you  suggest? 

8.  Was  the  learning  and  development  “process”  used  during  the  entire  program  (including  the  May  15-25  time) 
effective? 

Yes  Not  Really 

Why? 

9.  Describe  how  you  feel  about  the  “products”  you  have  developed,  their  purpose,  usefulness,  quality,  etc.? 

10.  Describe  teaching  and  learning  process  differences  that  you  will  implement  in  next  fall’s  course? 

11.  Do  you  feel  that  the  program  dates  worked... meaning  some  semester  time  and  some  summer  time;  our  time  was 
Vi  days  (9)  in  semester  and  Vx  days  (9)  in  May. 

Worked  well  Would  prefer  a different  schedule.  Describe  a preferred  schedule  for  18  days. 

12.  Strengths  of  the  program  overall. 

13.  Strengths  of  the  learning  and  development  process  overall. 

14.  Areas  to  improve  the  overall  program. 

15.  Areas  to  improve  the  learning  and  development  process  overall. 

16.  Dean  Vohra  would  like  the  your  Learning  Community  to  continue  and  actively  involve  each  of  you  together  to 
continue  to  learn,  share,  and  execute  research  on  teaching  and  learning.  At  this  point,  although  we  don’t  have  it 
well  defined,  are  you  willing  to  help  define  what  “continued  action”  together  means  and  then  continue  to 
participate? 

Yes  No,  probably  not 

17.  Did  you  learn  or  enhance  “other”  types  of  skills  through  the  program  process,  e.g.  computer  or  others? 

18.  General  Comments  about  the  overall  program: 
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GAPS  Analysis  Summary 


Standards  ABET-Engineering 


a.  apply 
math, 
science, 
engineering 

b.  design/conduct 
experiments; 
analyze,  interpret 
data 

c.  design  system,  component, 
process-given  constraints,  etc. 

d.  function  on 

interdisciplinary 

teams 

e.  identify,  formulate, 
solve  engineering 
problems 

f.  understand 
professional, 
ethical 

responsibility 

g.  ability  to 

communicate 

effectively 

h.  undst. 
impact  eng. 
Sol  global, 
economic, 
evnir., 
society 

i.  recognition  of 
need  for,  and 
ability  to  engage  in 
life-long  learning 

j.  Knowledge  in 

contemporary 

issues 

k.  ability  to  use  techniques,  skills, 
and  modern  engineering  tools 

Standards  ABET/' 


'AC/NAIT-Engineering  Technology  and  Industrial  Technology 


a.  mastery 
of 

knowledge, 

techniques, 

skills, 

modern 

tools 

b.  ability  to 

apply 

current 

knowledge; 

adapt  to 

emerging 

applications 

of  math, 

science, 

technology 

c.  ability  to 

conduct, 

analyze, 

interpret 

experiments; 

apply 

experimental 
results  to 
improve 
processes 

d.  ability  to 
apply 

creativity  in 
design  of 
systems, 
components, 
processes 

e.  ability 
to 

function 
effectively 
on  teams 

f.  ability 
to 

identify, 

analyze, 

solve 

technical 

problems 

g.  ability  to 
communicate 
effectively 
writing 

h.  ability  to 
communicate 
effectively 
orally 

recognize 
need  for, 
ability  to 
engage  in 
lifelong 
learning 

j.  ability  to 
understand 
professional, 
ethical,  social 
responsibilities 

k.  respect  for 

diversity; 

knowledge  of 

contemporary 

professional, 

societal, 

global  issues 

1.  commit 
to  quality, 
timeliness, 
continuous 
improvement 

m.  ability  to 
program 
computers 
and/or  use 
computer 
applications 
effectively 

n.  ability  to 

use  modern 

laboratory 

techniques, 

skills, 

equipment 

effectively 

o.  ability  to 
manage 
projects 
effectively 

p.  ability  to 

design, 

manipulate, 

manage 

industrial 

systems 

q.  ability 
to  manage 
or  lead 
personnel 
effectively 

NIU  General  Education 


Writing 

Speaking 

Listening 

Quantitative 

Reasoning 

Use  of  Resources-Technology 

Historical 
Development 
Of  Culture 

Significance  of 
Arts 

Cultural  Traditions 
Philosophical  Ideas 

Methods  in  Science 
Methods  in  Social  Science 

Interrelatedness 
Across  Disciplines 

Social  Responsibility 
Citizenship 
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Student  Learning  Outcomes  & Teaching  Styles 

Outcomes 

Command 

Practice 

Reciprocal 

Self-Check 

Inclusion 

Guided 

Discovery 

Convergent 

Discovery 

Divergent 

Production 

Learner 

Designed 

Learner 

Initiated 

Self 

Teaching 

Student  Learning  Objectives/Outcomes  & Learning  Styles 


Objectives 

Concrete  Experience 

Abstract  Conceptualization 

Active  Experimentation 

Reflective  Observation 
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Student  Learning  Objectives/Outcomes  & Bloom  & Dale 


Objectives 

Dale's  Cone 
Levels  P A 
A+ 

Knowledge 

Remember 

Comprehension 

Understand 

Application 

Apply 

Analysis 

Analyze 

Synthesis 

Evaluate 

Create 

Critical 
Thinking 
Level  LM  H 
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CEET  Initiative  on  Teaching  and  Learning 

Program  Eeedback-  (Einal  Eeedback,  end  of  development,  before  classroom  resesarch  semester ) - May  15-25 
1.  Were  the  sessions  May  15-25  worth  your  time  - worthwhile? 


Test  Analysis  and  Development  Review 

Yes 

Not  really 

Performance  Assessment  and  Rubies 

Yes 

Not  really 

Analyzing  all  your  assessments  by  Bloom 

Yes 

Not  really 

Consideration  of  “broader”  assessments  and  mapping  your  assessments 

Yes 

Not  really 

Teaching  Models,  including  Cooperative  Learning  and  Mapping  Analysis 

Yes 

Not  really 

The  review  of  components  for  a more  revealing  syllabus  for  the  students 

Yes 

Not  really 

The  consideration  of  Multicultural  aspects  of  courses. 

Yes 

Not  really 

The  consideration  of  grading. 

The  Research  Session 

Yes 

Not  really 

Why? 

2.  Erom  the  perspective  of  the  entire  program,  was  it  as  a “whole”  worth  your  time  - worthwhile? 

Yes  Not  really 

Why? 

3.  Would  you  recommend  the  overall  program  content  as  you  experienced  it  for  other  faculty  members? 

Yes  Not  really 

Why? 

4.  Would  you  recommend  that  other  faculty  members  get  the  opportunity  to  participate  in  this  program  with 
content  modifications? 

Yes  Not  really  - too  soon  to  determine 

What  changes  would  you  suggest? 


5.  Was  the  learning  and  development  “process”  used  during  the  entire  program  (including  the  May  15-25  time) 
effective? 

Yes  Not  Really 

Why? 

6.  Describe  how  you  feel  about  the  “products”  you  have  developed,  their  purpose,  usefulness,  quality,  etc.? 

7.  Describe  teaching  and  learning  process  differences  that  you  will  implement  in  next  fall’s  course? 

8.  Do  you  feel  that  the  program  dates  worked... meaning  some  semester  time  and  some  summer  time;  our  time  was 
1/2  days  (9)  in  semester  and  Y2  days  (9)  in  May. 

(7)  Worked  well  Would  prefer  a different  schedule.  Describe  a preferred  schedule  for  18  days. 

9.  Strengths  of  the  program  overall. 


10.  Strengths  of  the  learning  and  development  process  overall. 
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11.  Areas  to  improve  the  overall  program. 

12.  Areas  to  improve  the  learning  and  development  process  overall. 

13.  Dean  Vohra  would  like  the  your  Learning  Community  to  continue  and  actively  involve  each  of  you  together  to 
continue  to  learn,  share,  and  execute  research  on  teaching  and  learning.  At  this  point,  although  we  don’t  have  it 
well  defined,  are  you  willing  to  help  define  what  “continued  action”  together  mean  and  then  continue  to  participate? 

(7)  Yes  No,  probably  not 

14.  Did  you  learn  or  enhance  “other”  types  of  skills  through  the  program  process,  e.g.  computer  or  others? 

15.  General  Comments  about  the  overall  program: 
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CEET  Initiative  on  Teaching  and  Learning 
Final  Program  Feedback  (end  of  research  semester)-  December  15,  2006 

1.  Looking  back,  and  after  the  research  semester,  do  you  feel  that  the  faculty  development  sessions  were  worth  your 
time  - worthwhile? 

-Course  Analysis  (e.g.,  content  gap  analysis  & priority,  learning  styles,  teaching  models 


& styles,  standards  & learning  objectives/outcomes,  objectives  and  test  item  match. 
Bloom’s  Taxonomy,  Dale’s  Cone,  Critical  Thinking,  and  more) 

Yes 

Majority 

Not  really 

-Test  Analysis  and  Test  Development  Review 

Yes 

Majority 

Not  really 

- Performance  Assessment  and  Rubrics 

Yes 

Majority 

Not  really 

-Analyzing  all  your  assessments  by  Bloom 

Yes 

Majority 

Not  really 

- Consideration  of  “broader”  assessments  and  mapping  your  assessments 

Yes 

Majority 

Not  really 

-Teaching  Models,  including  Cooperative  Learning  and  Mapping  Analysis 

Yes 

Majority 

Not  really 

-The  review  of  components  for  a more  revealing  syllabus  for  the  students 

Yes 

Majority 

Not  really 

-The  review  and  consideration  of  Multicultural  aspects  of  courses. 

Yes 

Majority 

Not  really 

-The  review  and  consideration  of  grading. 

Yes 

Majority 

Not  really 

-The  educational  Research  Session 

Yes 

Majority 

Not  really 

-Overall,  was  the  faculty  development  program  worth  your  time  & “worth  while”? 

Yes(6) 

Majority(l)  Not  really 

****Why  or  Why  not  to  any  of  the  above  statements? 
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2.  Would  you  recommend  the  overall  program,  including  the  program  content  as  you  experienced  it,  for  other  faculty 
members? 

Yes,  definitely  Yes,  the  greater  majority  of  it  Some  of  it  Not  really 

****Why  or  why  not? 

3.  Looking  back  at  the  semester,  were  the  modifications  made  to  your  course,  or  the  changes  to  your  course, 
worthwhile  and  effective  with  students? 

Yes,  definitely- they  improved  the  course  and  instruction  Yes,  the  majority  of  them  improved  the  course  and 

instruction 

Yes,  some  of  them  improved  the  course  and  instruction 

Not  as  many  of  the  changes  were  as  valuable  as  I had  hoped  No,  very  few  of  them  were  successful  changes 

****Why  or  Why  not? 

4.  Were  the  teaching  and  learning  materials  developed  during  the  faculty  development  sessions  effective  when  used 
with  students  during  the  research  semester? 

Yes,  definitely  Yes,  greater  the  majority  of  them  Yes,  some  of  them 

Not  as  many  of  the  materials  were  as  valuable  as  I had  hoped  No,  very  few  of  the  materials  were  successful 

****Why  or  Why  not? 

5.  Describe  how  you  feel  about  the  “products”  you  have  developed,  their  purpose,  usefulness,  quality,  etc.? 

Analysis  products  - gaps  analysis,  teaching  models  and  styles  analysis,  learning  styles  analysis,  course  content  Analysis 
Syllabus  - 

Tests  - 

Test  Analyses  - 
Performance  Assessments  - 
Rubrics  - 

Choices  of  teaching  models  and  processes  - 
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6.  Was  the  Research  Semester,  performing  experimental  classroom  research  with  students,  worth  your  time  and 
“worthwhile”? 

Yes,  very  valuable,  beneficial  in  the  following  ways: 

provided  evidence  of  the  benefit  of  the  course,  teaching,  learning,  content,  test,  etc.  gaps  analyses  and  what  was 

learned  from  them  and  developed  as  a result  of  identifying  the  gaps  through  the  analysis  processes 

provided  opportunity  to  evaluate  and  see  benefits  of  new  teaching  and  learning  or  educational  products 

provided  opportunity  to  evaluate  and  see  benefits  of  new  teaching  and  learning  processes 

provided  opportunity  to  evaluate  and  see  benefits  of  new  teaching  models  and  styles 

provided  insight  into  student  learning 

provided  insight  about  my  teaching 

provided  opportunity  for  a first  attempt  at  educational  research  - scholarship  of  teaching 

identify  and  list  others: 

Not  really  as  valuable  as  I had  hoped:  describe  why  for  each  item  below: 

provided  evidence  of  the  benefit  of  the  course,  teaching,  learning,  content,  test,  etc.  gaps  analyses  and  what  was 

learned  from  them  and  developed  as  a result  of  identifying  the  gaps  through  the  analysis  processes 

provided  opportunity  to  evaluate  and  see  benefits  of  new  teaching  and  learning  or  educational  products 

provided  opportunity  to  evaluate  and  see  benefits  of  new  teaching  and  learning  processes 

provided  opportunity  to  evaluate  and  see  benefits  of  new  teaching  models  and  styles 

provided  insight  into  student  learning 

provided  insight  about  my  teaching 

provided  opportunity  for  a first  attempt  at  educational  research  - scholarship  of  teaching 

identify  and  list  others: 

7.  Would  you  recommend  to  other  faculty  members  that  they  begin  to  engage  in  research  on  teaching  and  learning? 

Yes,  definitely  Yes,  definitely,  but  after  participating  in  the  faculty  development  to  prepare  them 

No,  not  really 

*****Why?  Specifically,  what  would  keep  you  from  recommending  that  others  engage  in  classroom  research  on 
teaching  and  learning?  Please  describe  in  detail. 

8.  From  the  perspective  of  the  entire  program,  faculty  development,  the  development  of  course  and  classroom 
materials,  and  the  educational  research  semester,  was  the  entire  program  - beginning  with  analysis  through  classroom 
research-  as  a “whole”  program,  worth  your  time  - worthwhile? 

Yes  The  greater  majority  Some  of  it  Really,  Not  much 

****Specifically,  Why  or  Why  not? 
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9.  Would  you  recommend  that  other  faculty  members  get  the  opportunity  to  participate  in  this  “whole”  program, 
including  faculty  development,  course  development,  and  classroom  research  ? 

Yes,  definitely  Yes,  with  a few  content  changes  Yes,  with  many  content  changes?  No,  not  really 

10.  Specifically,  what  content  changes  would  you  suggest? 

11.  Was  the  learning  and  development  “process”  used  during  the  entire  program  (Oct.OS-research,  Dec.06)  effective? 

Yes,  definitely  Yes,  with  a few  process  changes  Yes,  with  many  process  changes  No,  not  really 

No  response 

12.  Specifically,  what  process  changes  would  you  suggest?  ****Why? 

13.  Specifically,  what  about  the  program,  overall,  would  keep  you  from  recommending  it  to  other  faculty  members? 

14.  Identify  and/or  describe  the  teaching  and  learning  changes  that  you  implemented  in  the  research  semester’s  course: 

Improved  priority  of  course  content 

New  syllabus  with  many  new  components 

Clear  learning  objectives/outcomes  tied  to  ABET/NAIT  standards 

Learning  Style  Inventory,  e.g.  Kolb,  Felder,  other 

New  teaching  models 

Identify,  for  example: 

Small  groups 
One  minute  papers 

New  teaching  styles 

New  objective  tests 

New  performance  assessments/Rubrics 

New  grading  criteria  - clear  and  pre-determined,  no  curving  of  grades,  or  last  minute  non-criteria-based 

judgments,  etc. 

Better  alignment  of  syllabus,  teaching,  and  assessment. 

Identify  and  list  others  specific  to  you. 

15.  Do  you  feel  that  the  program  dates  worked... recall  that  some  time  was  spent  during  the  regular  semesters  and 
some  time  during  the  summer;  specifically  there  was  y^  or  9 days  across  the  regular  semesters  and  Vx  or  9 days  in  May, 
plus  2 fall  meetings,  and  the  final  meeting? 

Worked  well  Would  prefer  a different  schedule 

No  response 

****Describe  a preferred  schedule  for  18  days,  plus  several  short  meetings 

16.  Describe  the  strengths  of  the  overall  program  content. 
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17.  Describe  the  strengths  of  the  overall  learning  and  development  process. 

18.  Describe  the  areas  you  would  like  to  see  improvements  in  regarding  the  overall  program  content  AND  specify  the 
desired  improvements. 

19.  Describe  the  areas  to  you  would  like  to  see  improvements  in  related  to  the  overall  learning  and  development  process 
AND  specify  the  desired  improvements. 

20.  Dean  Vohra  would  like  your  Learning  Community  to  continue  and  actively  involve  each  of  you  together  to  continue 
to  learn,  share,  and  execute  research  on  teaching  and  learning.  At  this  point,  although  we  don’t  have  it  well  defined,  are 
you  willing  to  help  define  what  “continued  action”  together  means  and  then  continue  to  participate? 

Yes,  definitely  No,  probably  not  - Why? 

It  will  depend  upon....?  Describe  for  you. 

21.  Other  General  Comments  about  the  overall  program: 
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Tech  496  TAC-NAIT  Learning  Outcome- General  Education  Example 
ABET/TAC/NAIT  Bloom's  Knowledge  Dale’s  Student  Learning 

Engineering  & Technology  Dimension  Cone  Outcomes 


Bloom's  Cognitive  Process  Dimension 

Knowledge  Comprehension  Application  Analysis  Synthesis 

Remember  Understand  Apply  Analyze  Evaluate 


A.  Mastery  of  knowledge,  techniques, 
skills,  modern  tools  of  disciplines. 


J Factual  Knowledge:  Active 

J Conceptual  Knowledge: 

J Procedural  Knowledge: 


6.  To  demonstrate  effective  project 

a.  planning 

b.  initiation 

c.  execution 

d.  termination 


J Meta-cognitive  Knowledge: 


B.  Apply  current  knowledge  and  J Factual  Knowledge:  Active 

adapt  to  emerging  applications 

of  math,  science,  engineering,  and  J Conceptual  Knowledge: 

technology. 

J Procedural  Knowledge: 

J Meta-cognitive  Knowledge: 


NIU  Gen  Ed  Goals  - Students: 
a.iii.  perform  basic  computations, 
display  facility  with  use  of  formal 
and  quantitative  reasoning  analysis 
and  problem  solving,  and  interpret 
mathematical  models  and 
statistical  information. 


8.  To  integrate  mathematics,  the  sciences, 
communication,  management,  technical, 
and  technological  knowledge  and  skills 
to  accomplish  team  and  project  objectives. 


C.  Conduct,  analyze,  and  interpret  Factual  Knowledge: 

experiments;  apply  experimental 

results  to  improve  processes.  Conceptual  Knowledge: 


Gen  Ed  Goals  - Students;  Procedural  Knowledge: 

b.  develop  an  ability  to  use  modes 

of  inquiry  across  a variety  of  Meta-cognitive  Knowledge: 

disciplines  in  the  humanities  and 

the  arts,  the  physical  sciences 

and  mathematics,  and  social 

sciences. 

b.iv.  demonstrate  an  ability  to 
use  scientific  methods  and 
theories  to  understand  the 
phenomena  studied  in  the 
natural  and  social  sciences. 


NA 
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Evaluate 

Create 


ABET/TAC/NAIT  Bloom's  Knowledge  Dale’s  Student  Learning 

Engineering  & Technology  Dimension  Cone  Outcomes 


Bloom's  Cognitive  Process  Dimension 

Knowledge  Comprehension  Application  Analysis  Synthesis 

Remember  Understand  Apply  Analyze  Evaluate 


D.  ability  to  apply  creativity  in  the  J Factual  Knowledge:  Active 

design  of  systems,  components, 

or  processes  appropriate  to  J Conceptual  Knowledge: 

program  objectives. 

J Procedural  Knowledge: 


NIU  Gen  Ed  Goals  - Students; 
c.  develop  an  understanding  of 
the  relatedness  of  various 
disciplines  by  integrating 
knowledge  from  several 
disciplines  and  applying  that 
knowledge  to  an  understanding 
of  important  problems  and 


J Meta-cognitive  Knowledge: 


E.  Function  effectively  on  teams. 


NIU  Gen  Ed  Goals-Students; 
b.iii.  demonstrate  a knowledge  of 
cultural  traditions  and 
philosophical  ideas  that  have 
shaped  societies,  civilizations,  and 
human  self-conceptions, 
d.  develop  social  responsibility  and 
preparation  for  citizenship  through 
global  awareness,  environmental 
sensitivity,  and  an  appreciation  of 
cultural  diversity. 


J Factual  Knowledge:  Active 

J Conceptual  Knowledge: 

J Procedural  Knowledge 
J Meta-cognitive  Knowledge: 


8.  To  integrate  mathematics,  the  sciences, 
communication,  management,  technical, 
technological  knowledge  and  skills  to 
accomplish  team  and  project  objectives. 

a.  design  a vehicle  to  technical  specifications 

b.  build  the  vehicle  to  technical  specifications 

c.  solve  technical  problems  associated  with 
design,  construction,  and  evaluation 

d.  test  and  evaluate  vehicle  against  technical 
specifications 


1.  To  identify  and  describe  major  problems. 

Issues,  concerns,  and  solutions  that  relate  to 
projects,  PM,  Pteams,  and  Pleaders,  international 
projects  and  multi-cultural  teams. 

2.  To  identify  and  describe  best  practices  for  managing 
projects  and  leading  teams,  including  international 
projects  and  multi-cultural  teams. 

3.  To  perform  effectively  on  a project  team 
(multi-cultural  when  possible) 

a.  To  engage  in  conflict  resolution  to  resolve 
team  issues. 


Evaluate 

Create 
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ABET/TAC/NAIT 
Engineering  & Technology 


Bloom's  Knowledge  Dale’s  Student  Learning  Outcomes 

Dimension  Cone 


F.  Identify,  analyze,  and  solve 
technical  problems. 


Gen  Ed  Goals  - Students: 
b.  develop  an  ability  to  use  modes  of 
inquiry  across  a variety  of  disciplines 
in  the  humanities  and  the  arts,  the 
physical  sciences  and  mathematics, 
and  social  sciences, 
b.iv.  demonstrate  an  ability  to  use 
scientific  methods  and  theories  to 
understand  the  phenomena  studied  in 
the  natural  and  social  sciences. 


J Factual  Knowledge:  Active 

J Conceptual  Knowledge: 

J Procedural  Knowledge: 

J Meta-cognitive  Knowledge: 


8.  To  integrate  mathematics,  the  sciences, 
communication,  management,  technical, 
technological  knowledge  and  skills  to 
accomplish  team  and  project  objectives. 

a.  design  a vehicle  to  technical  specifications 

b.  build  the  vehicle  to  technical  specifications 

c.  solve  technical  problems  associated  with 
design,  construction,  and  evaluation 

d.  test,  evaluate  vehicle  against  technical 
specifications. 


G.  Communicate  effectively  in  writing. 


NIU  Gen  Ed  Goals-  Students: 
a.  develop  habits  of  writing, 
speaking,  and  reasoning  necessary 
for  continued  learning, 
a.i.  communicate  clearly  in  written 
English,  demonstrating  ability  to 
comprehend,  analyze,  and 
interrogate  critically. 


J Factual  Knowledge:  Active 

J Conceptual  Knowledge: 

J Procedural  Knowledge: 


9.  To  design,  develop,  prepare,  and  deUver 

a.  executive  team  presentation 

b.  team  portfolio 

c.  team  website 


J Meta-cognitive  Knowledge: 
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Bloom's  Cognitive  Process  Dimension 

Knowledge  Comprehension  Application  Analysis  Synthesis  Evaluate 

Remember  Understand  Apply  Analyze  Evaluate  Create 


ABET/TAC/NAIT 
Engineering  & Technology 


Bloom's  Knowledge  Dale’s 
Dimension  Cone 


Student  Learning  Outcomes 


Bloom's  Cognitive  Process  Dimension 

Knowledge  Comprehension  Application  Analysis  Synthesis 

Remember  Understand  Apply  Analyze  Evaluate 


Evaluate 

Create 


H.  Communicate  effectively  orally. 


NIU  Gen  Ed  Goals  - Students: 
a.  develop  habits  of  writing, 
speaking,  and  reasoning 
necessary  for  continued 
learning. 

a.i.  communicate  clearly  in 
written  English,  demonstrating 
ability  to  comprehend,  analyze, 
and  interrogate  critically, 
aii.  communicate  in  a manner 
that  unites  theory,  criticism,  and 
practice  in  speaking  & writing. 


I.  Recognize  the  need  for,  and  an 
ability  to  engage  in  life  long  learning. 


NIU  Gen  Ed  Goals  - Students: 
a.iv.  are  able  to  access  and  use 
various  information  sources. 
Internet,  text,  and  field  case. 


J.  Understand  professional,  ethical, 
and  social  responsibilities. 


NIU  Gen  Ed  Goals  - Students: 
d.  develop  social  responsibility  and 
preparation  for  citizenship  through 
global  awareness,  environmental 
sensitivity,  and  an  appreciation  of 
cultural  diversity. 


J Factual  Knowledge:  Active 

J Conceptual  Knowledge: 

J Procedural  Knowledge: 

J Meta-cognitive  Knowledge: 


7.  To  demonstrate  effective  project 

a.  planning 

b.  initiation 

c.  execution 

d.  termination 

9.  To  design,  develop,  prepare,  and  deliver 

a.  team  presentation 

b.  team  portfolio 

c.  team  website 


J Factual  Knowledge: 

J Conceptual  Knowledge: 
Procedural  Knowledge: 


A 

Intermdeidatel.  To  identify  and  describe  major  problems, 

Active  issues,  concerns,  and  solutions  that  relate  to  / 
projects,  project  management,  project  teams,  / 
and  project  leaders,  including  international  ^ 
projects  and  multi-cultural  teams. 


Meta-cognitive  Knowledge:  2.  To  identify  and  describe  best  practices  for 

managing  projects  and  leading  teams,  including 
international  projects  and  multi-cultural  teams. 


J Factual  Knowledge:  Active 

J Conceptual  Knowledge: 


5.  To  exhibit  leadership  by  engaging  in 
a team  community  service  project. 


J Procedural  Knowledge: 


J Meta-cognitive  Knowledge: 
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ABET/TAC/NAIT  Bloom's  Knowledge  Dale’s 

Engineering  & Technology  Dimension  Cone 


K.  Respect  for  diversity  and  a knowledge  VF actual  Knowledge:  Active 

of  contemporary  professional,  societal, 

and  global  issues.  ^Conceptual  Knowledge: 

^Procedural  Knowledge: 
^Meta-cognitive  Knowledge: 


L.  Commitment  to  quality,  timeliness,  ^Factual  Knowledge:  Active 

and  continuous  improvement. 

^Conceptual  Knowledge: 
^Procedural  Knowledge: 
^Meta-cognitive  Knowledge: 


NIU  Gen  Ed  Goals  - Students: 
d.  develop  social  responsibility  and 
preparation  for  citizenship  through 
global  awareness,  environmental 
sensitivity,  and  an  appreciation  of 
cultural  diversity. 


Student  Learning  Outcomes  Bloom's  Cognitive  Process  Dimension 

Knowledge  Comprehension  Application 

Remember  Understand  Apply 


3.  To  perform  effectively  on  a project  team 
(multi-cultural  when  possible), 
a.  To  engage  in  conflict  resolution  to  resolve 
team  issues. 


Analysis 

Analyze 


Synthesis 

Evaluate 


Evaluate 

Create 


6.  To  demonstrate  effective  project 

a.  planning 

b.  initiation 

c.  execution 

d.  termination 

3.  To  perform  effectively  on  a project  team 
(multicultural  when  possible) 

a.  To  engage  in  conflict  resolution  to  resolve  team  issues. 


8.  To  integrate  mathematics,  the  sciences 
communication,  management,  technical 
team  and  project  objectives. 

a.  design  a vehicle  to  technical  specifications 

b.  build  the  vehicle  to  technical  specifications 

c.  solve  technical  problems  associated  with 
design,  construction,  and  evaluation 

d.  test  and  evaluate  vehicle  against  technical  specifications 


M.  Ability  to  program  computers 
and/or  utilize  computer  applications 
effectively. 


VF actual  Knowledge:  Active 

^Conceptual  Knowledge: 
^Procedural  Knowledge: 
^Meta-Cognitive  Knowledge: 


6.  To  demonstrate  effective  project 

a.  planning 

b.  initiation 

c.  execution 

d.  termination 
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ABET/TAC/NAIT 
Engineering  & Technology 


N.  Ability  to  use  modern  laboratory 
techniques,  skills,  and/or  equipment 
effectively. 


Bloom's  Knowledge  Dale’s 
Dimension  Cone 


Student  Learning  Outcomes  Bloom's  Cognitive  Process  Dimension 

Knowledge  Comprehension  Application  Analysis  Synthesis  Evaluate 

Remember  Understand  Apply  Analyze  Evaluate  Create 


/Factual  Knowledge:  Active 

/Conceptual  Knowledge: 
/Procedural  Knowledge: 
/Meta-Cognitive  Knowledge: 


8.  To  integrate  mathematics,  the  sciences 
communication,  management,  technical  and  — 
technological  knowledge  and  skills  to  accomplislN. 
team  and  project  objectives. 

a.  design  a vehicle  to  technical  specifications  , 

b.  build  the  vehicle  to  technical  specifications 

c.  solve  technical  problems  associated  with  project 
design,  construction,  and  evaluation 

d.  test  and  evaluate  vehicle  against  technical  specifications 
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ABET/TAC/NAIT 
Engineering  & Technology 


Bloom's  Knowledge  Dale’s 
Dimension  Cone 


Student  Learning  Outcomes 


Bloom's  Cognitive  Process  Dimension 

Knowledge  Comprehension  Application  Analysis  Synthesis 

Remember  Understand  Apply  Analyze  Evaluate 


Evaluate 

Create 


O.  Ability  to  manage  projects 
effectively. 


NIU  Gen  Ed  Goals  - Students: 
c.  develop  an  understanding  of 
the  interrelatedness  of  various 
disciplines  by  integrating 
knowledge  from  several 
disciplines  and  applying  that 
knowledge  to  an  understanding 
of  important  problems  and 


yp actual  Knowledge:  Active 

i/Conceptual  Knowledge: 
^Procedural  Knowledge: 
i/Meta-cognitive  Knowledge: 


1.  To  identify  an  describe  major  problems,  issues,  concerns 
and  solutions  that  relate  to  projects,  Pmanagement,  P teams, 
and  P leaders,  including  international  projects  and  leading 
Multi-cultural  teams. 


2.  To  identify  and  describe  best  practices  for  managing  projects 
and  leading  teams,  including  international  projects  and  multi- 
cultural teams. 


3.  To  perform  effectively  on  a project  team  (multi-cultural  team 
when  possible). 


Active  4.  To  prepare  the  team  for  project  work  by 

a.  developing  a team  operations  manual 

b.  developing  a peer  and  team  assessment  system 

c.  creating  the  team  organization  and  process 

d.  developing  a team  project  plan 

6.  To  demonstrate  project 

a.  planning 

b.  initiation 

c.  execution 

d.  termination 

e.  problem  solving 


7.  To  demonstrate  effective  use  of  project  management  techniques 
and  tools  in  the  management  of  a technical  project. 

a.  the  development  of  a project  plan 

b.  use  of  MS  Project 

c.  use  of  appropriate  financial  planning  and  operations  procedures 

d.  use  of  appropriate  procurement  procedures 

e.  scheduling  techniques 

f.  use  of  the  MACE  process 


9.  To  design,  develop,  prepare,  and  deliver 

a.  executive  team  presentation 

b.  team  portfolio 

c.  team  website 
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ABET/TAC/NAIT  Bloom's  Knowledge  Dale’s  Student  Learning  Outcomes 

Engineering  & Technology  Dimension  Cone 


P.  Ability  to  design,  manipulate,  Factual  Knowledge; 

and  manage  industrial  systems. 

Conceptual  Knowledge: 
Procedural  Knowledge: 
Meta-cognitive  Knowledge: 


NIU  Gen  Ed  Goals  - Students: 
c.  develop  an  understanding  of 
the  interrelatedness  of  various 
disciplines  by  integrating 
knowledge  from  several 
disciplines  and  applying  that 
knowledge  to  an  understanding 
of  important  problems  and 


NA 


Bloom's  Cognitive  Process  Dimension 

Knowledge  Comprehension  Application 

Remember  Understand  Apply 


Analysis  Synthesis 
Analyze  Evaluate 


Evaluate 

Create 


Q.  Ability  to  manage  or  lead  personnel 
effectively. 


^Factual  Knowledge:  Active  3.  To  perform  effectively  on  a project  team 

(multi-cultural  when  possible). 

^Conceptual  Knowledge:  a.  To  engage  in  conflict  resolution  to  resolve 

team  issues. 

^Procedural  Knowledge:  b.  To  engage  in  the  leadership  of  the  team, 

team  members,  or  work  package  sub-team 

VMeta-cognitive  Knowledge:  members. 


4.  The  team  will  prepare  project  work  by 

a.  developing  a team  operations  manual 

b.  developing  a peer  and  team  assessment  system 

c.  creating  the  team  organization  and  process 

d.  developing  a team  project  plan 


5.  To  exhibit  leadership  while  engaged  in  a 
team  community  service  project 

a.  plan 

b.  initiate 

c.  execute 

d.  terminate 

e.  report 

Addition:  additional  educational  outcomes  articulated  by  the  overall  program  **  See  in  text  boxes  above  - NIU  General  Education  Goals 
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ABET  Engineering  Objeetive  Worksheet 


ABET  Engineering  Outcomes 

Bloom's  Knowledge 
Dimension 
J on  left  side  of  type  of 
knowledge  that  applies  to 
outcome 

Dale’s  Cone  of 
Learning 

Student  Learning 
Objectives 

Bloom’s  Cognitive  Process  Dimension 

Passive 

Active 

Knowledge 

Remember 

Comprehension 

Understand 

Application 

Apply 

Analysis 

Analyze 

Synthesis 

Evaluate 

Evaluate 

Create 

A 

s 

. Apply  knowledge  of  math, 
cience,  engineering 

NIU  Gen  Ed  Goals  - 
Students: 

a.  Hi.  perform  basic 
computations,  display 
facility  with  use  of 
formal  and  quantitative 
reasoning  analysis  and 
problem  solving,  and 
interpret  mathematical 
models  and  statistical 
information. 

Factual  Knowledge 
Conceptual  Knowledge 
Procedural  Knowledge 
Meta-Cognitive  Knowledge 

B 

ai 

Design  and  conduct  experimen 
lalyze  and  interpret  data 

NIU  Gen  Ed  Goals  - Students: 
a.iii.  perform  basic 
computations,  display 
facility  with  use  of  formal 
and  quantitative  reasoning 
analysis  and  problem 
solving,  and  interpret 
mathematical  models  and 
statistical  information. 

Is; 

Factual  Knowledge 
Conceptual  Knowledge 
Procedural  Knowledge 
Meta-Cognitive  Knowledge 

1 


ABET  Engineering  Outcomes 


Bloom's  Knowledge 
Dimension 
J on  left  side  of  type  of 
knowledge  thet  epplies  to 
outcome 


C.  Design  a system,  component, 
process  to  meet  desired  needs  within 
realistic  constraints  (e.g.,  economic, 
environmental,  social,  political, 
ethical,  health,  safety, 
mannfacturability,  & snstainabiUty). 


Factual  Knowledge 
Conceptual  Knowledge 
Procedural  Knowledge 


Dale’s  Cone  of  Student  Learning 

Learning Objectives 

Passive 

Active 


Bloom’s  Cognitive  Process  Dimension 


Knowledge 

Remember 


Comprehension 

Understand 


Application  Analysis 
Apply  Analyze 


Synthesis 

Evalnate 


Evaluate 

Create 


Meta-Cognitive  Knowledge 


NIU  Gen  Ed  Goals  - 
Students: 

c.  develop  an  understanding 
of  the  relatedness  of  various 
disciplines  by  integrating 
knowledge  from  several 
disciplines  and  applying  that 
knowledge  to  an 
understanding  of  important 
problems  and  issues. 


2 


ABET  Engineering  Outcomes 

Bloom's  Knowledge 
Dimension 
J on  left  side  of  type  of 
knowledge  thet  epplies  to 
outcome 

Dale’s  Cone  of 
Learning 

Student  Learning 
Objectives 

Bloom’s  Cognitive  Process  Dimension 

Passive 

Active 

Knowledge 

Remember 

Comprehension 

Understand 

Application 

Apply 

Analysis 

Analyze 

Synthesis 

Evaluate 

Evaluate 

Create 

D.  Function  on  multi-disdplinai 
teams. 

NIU  Gen  Ed  Goals- 
Students: 
b.iii.  demonstrate  a 
knowledge  of  cultural 
traditions  and  philosophical 
ideas  that  have  shaped 
societies,  civilizations,  and 
human  self-conceptions, 
d.  develop  social 
responsibility  and 
preparation  for  citizenship 
through  global  awareness, 
environmental  sensitivity, 
and  an  appreciation  of 
cultural  diversity. 

fy 

Factual  Knowledge 
Conceptual  Knowledge 
Procedural  Knowledge 
Meta-Cognitive  Knowledge 

3 


ABET  Engineering  Outcomes 


Bloom's  Knowledge 
Dimension 
J on  left  side  of  type  of 
knowledge  thet  epplies  to 


Dale’s  Cone  of  Student  Learning 

Learning Objectives 

Passive 

Active 


Bloom’s  Cognitive  Process  Dimension 


Knowledge 

Remember 


Comprehension 

Understand 


Application  Analysis 
Apply  Analyze 


Synthesis 

Evalnate 


Evaluate 

Create 


outcome 


E.  Identify,  formulate,  and  solve  Factual  Knowledge 

engineering,  problems 

Conceptual  Knowledge 


Gen  Ed  Goals  - Students: 
b.  develop  an  ability  to  use 
modes  of  inquiry  across  a 
variety  of  disciplines  in  the 
humanities  and  the  arts,  the 
physical  sciences  and 
mathematics,  and  social 
sciences. 

b.iv.  demonstrate  an  ability 
to  use  scientific  methods  and 
theories  to  understand  the 
phenomena  studied  in  the 
natural  and  social  sciences. 


Procedural  Knowledge 
Meta-Cognitive  Knowledge 


4 


ABET  Engineering  Outcomes 

Bloom's  Knowledge 
Dimension 
J on  left  side  of  type  of 
knowledge  thet  epplies  to 
outcome 

Dale’s  Cone  of 
Learning 

Student  Learning 
Objectives 

Bloom’s  Cognitive  Process  Dimension 

Passive 

Active 

Knowledge 

Remember 

Comprehension 

Understand 

Application 

Apply 

Analysis 

Analyze 

Synthesis 

Evaluate 

Evaluate 

Create 

F.  Understand  professional  and 
ethical  responsibility 

NIU  Gen  Ed  Goals  - Students: 
d.  develop  social  responsibility  and 
preparation  for  citizenship  through 
global  awareness,  environmental 
sensitivity,  and  an  appreciation  of 
cultural  diversity. 

Factual  Knowledge 
Conceptual  Knowledge 
Procedural  Knowledge 
Meta-Cognitive  Knowledge 

G.  Communicate  effectively 

NIU  Gen  Ed  Goals-  Students: 
a.  develop  habits  of  writing, 
speaking,  and  reasoning  necessary 
for  continued  learning, 
a.i.  communicate  clearly  in  written 
English,  demonstrating  ability  to 
comprehend,  analyze,  and 
interrogate  critically. 

Factual  Knowledge 
Conceptual  Knowledge 
Procedural  Knowledge 
Meta-Cognitive  Knowledge 

5 


ABET  Engineering  Outcomes 

Bloom's  Knowledge 
Dimension 
J on  left  side  of  type  of 
knowledge  thet  epplies  to 
outcome 

Dale’s  Cone  of 
Learning 

Student  Learning 
Objectives 

Bloom’s  Cognitive  Process  Dimension 

Passive 

Active 

Knowledge 

Remember 

Comprehension 

Understand 

Application 

Apply 

Analysis 

Analyze 

Synthesis 

Evaluate 

Evaluate 

Create 

H.  understand  impact  of  engineering 
solutions  in  a global  economic, 
environmental,  societal  context 

NIU  Gen  Ed  Goals  - Students: 
d.  develop  social  responsibility  and 
preparation  for  citizenship  through 
global  awareness,  environmental 
sensitivity,  and  an  appreciation  of 
cultural  diversity. 

Factual  Knowledge 
Conceptual  Knowledge 
Procedural  Knowledge 
Meta-Cognitive  Knowledge 

I.  Recognize  the  need  for,  and  have 
capability  to  engage  in  life  long 
learning. 

NIU  Gen  Ed  Goals  - Students: 
a.iv.  are  able  to  access  and  use 
various  information  sources. 
Internet,  text,  and  field  case. 

Factual  Knowledge 
Conceptual  Knowledge 
Procedural  Knowledge 
Meta-Cognitive  Knowledge 

J.  knowledge  of  contemporary  issues 

NIU  Gen  Ed  Goals  - Students: 
d.  develop  social  responsibility  and 
preparation  for  citizenship  through 
global  awareness,  environmental 
sensitivity,  and  an  appreciation  of 
cultural  diversity. 

Factual  Knowledge 
Conceptual  Knowledge 
Procedural  Knowledge 
Meta-Cognitive  Knowledge 

6 


ABET  Engineering  Outcomes 

Bloom's  Knowledge 
Dimension 
J on  left  side  of  type  of 
knowledge  thet  epplies  to 
outcome 

Dale’s  Cone  of 
Learning 

Student  Learning 
Objectives 

Bloom’s  Cognitive  Process  Dimension 

Passive 

Active 

Knowledge 

Remember 

Comprehension 

Understand 

Application 

Apply 

Analysis 

Analyze 

Synthesis 

Evaluate 

Evaluate 

Create 

K.  use  techniques,  skills,  and  modern 
engineering  tools  necessary  for 
engineering  practice 

Factual  Knowledge 
Conceptual  Knowledge 
Procedural  Knowledge 
Meta-Cognitive  Knowledge 

Addition:  Additional  educational  outcomes  -NIU  General  Education  Goals  - articulated  by  the  overall  program 
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ABET/TAC/NAIT  Engineering 
& Technology  Outcomes 

Bloom's  Knowledge 
Dimension 
J an  left  side  of  type  of 
knowledge  thet  applies  to 
nutcnme 

Dale’s  Cone  of 
Learning 

Student  Learning 
Objectives 

Bloom’s  Cognitive  Process  Dimension 

Passive 

Active 

Knowledge 

Remember 

Comprehension 

Understand 

Application 

Apply 

Analysis 

Analyze 

Synthesis 

Evaluate 

Evaluate 

Create 

A.  Mastery  of  knowledge,  techniques, 
skills,  modern  tools  of  disciplines. 

Factual  Knowledge 
Conceptual  Knowledge 
Procedural  Knowledge 
Meta-Cognitive  Knowledge 

B.  Apply  current  knowledge  and 
adapt  to  emerging  applications  of 
math,  science,  engineering,  and 
technology. 

Factual  Knowledge 
Conceptual  Knowledge 
Procedural  Knowledge 
Meta-Cognitive  Knowledge 

NIU  Gen  Ed  Goals  - Students: 
a.  Hi.  perform  basic 
computations,  display 
facility  with  use  of  formal 
and  quantitative  reasoning 
analysis  and  problem 
solving,  and  interpret 
mathematical  models  and 
statistical  information. 

1 


ABET/TAC/NAIT  Engineering 
& Technology  Outcomes 

Bloom's  Knowledge 
Dimension 
J an  left  side  of  type  of 
knowledge  thet  applies  to 
□utcnme 

Dale’s  Cone  of 
Learning 

Student  Learning 
Objectives 

Bloom’s  Cognitive  Process  Dimension 

Passive 

Active 

Knowledge 

Remember 

Comprehension 

Understand 

Application 

Apply 

Analysis 

Analyze 

Synthesis 

Evaluate 

Evaluate 

Create 

C 

ex 

re 

Conduct,  analyzes,  and  interpr 
periments;  apply  experimental 
suits  to  improve  processes. 

Gen  Ed  Goals  - Students: 
b.  develop  an  ability  to  use 
modes  of  inquiry  across  a 
variety  of  disciplines  in  the 
humanities  and  the  arts,  the 
physical  sciences  and 
mathematics,  and  social 
sciences. 

b.iv.  demonstrate  an  ability 
to  use  scientific  methods  and 
theories  to  understand  the 
phenomena  studied  in  the 
natural  and  social  sciences. 

ets 

Factual  Knowledge 
Conceptual  Knowledge 
Procedural  Knowledge 
Meta-Cognitive  Knowledge 

2 


ABET/TAC/NAIT  Engineering  Bloom's  Knowledge 
& Technology  Outcomes  Dimension 

J an  left  side  of  type  of 
knowledge  thet  applies  to 


Dale’s  Cone  of  Student  Learning 

Learning Objectives 

Passive 

Active 


Bloom’s  Cognitive  Process  Dimension 


Knowledge 

Remember 


Comprehension 

Understand 


Application  Analysis 
Apply  Analyze 


Synthesis 

Evaluate 


Evaluate 

Create 


□utcnme 


D.  Ability  to  apply  creativity  in  the 
design  of  systems,  components,  or 
processes  appropriate  to  program 
objectives. 


Factual  Knowledge 
Conceptual  Knowledge 


Procedural  Knowledge 


NIU  Gen  Ed  Goals  - 
Students: 

c.  develop  an  understanding 
of  the  relatedness  of  various 
disciplines  by  integrating 
knowledge  from  several 
disciplines  and  applying  that 
knowledge  to  an 
understanding  of  important 
problems  and  issues. 


Meta-Cognitive  Knowledge 


3 


ABET/TAC/NAIT  Engineering 
& Technology  Outcomes 

Bloom's  Knowledge 
Dimension 
J an  left  side  of  type  of 
knowledge  thet  applies  to 
□utcnme 

Dale’s  Cone  of 
Learning 

Student  Learning 
Objectives 

Bloom’s  Cognitive  Process  Dimension 

Passive 

Active 

Knowledge 

Remember 

Comprehension 

Understand 

Application 

Apply 

Analysis 

Analyze 

Synthesis 

Evaluate 

Evaluate 

Create 

E. 

Function  effectively  on  teams. 

NIU  Gen  Ed  Goals- 
Students: 
b.iii.  demonstrate  a 
knowledge  of  cultural 
traditions  and  philosophical 
ideas  that  have  shaped 
societies,  civilizations,  and 
human  self-conceptions, 
d.  develop  social 
responsibility  and 
preparation  for  citizenship 
through  global  awareness, 
environmental  sensitivity, 
and  an  appreciation  of 
cultural  diversity. 

Factual  Knowledge 
Conceptual  Knowledge 
Procedural  Knowledge 
Meta-Cognitive  Knowledge 

4 


F.  Identify,  analyze,  and  solve 
technical  problems. 

Gen  Ed  Goals  - Students: 
b.  develop  an  ability  to  use 
modes  of  inquiry  across  a 
variety  of  disciplines  in  the 
humanities  and  the  arts,  the 
physical  sciences  and 
mathematics,  and  social 
sciences. 

b.iv.  demonstrate  an  ability 
to  use  scientific  methods  and 
theories  to  understand  the 
phenomena  studied  in  the 
natural  and  social  sciences. 

Factual  Knowledge 
Conceptual  Knowledge 
Procedural  Knowledge 
Meta-Cognitive  Knowledge 

G.  Communicate  effectively  in 
writing. 

NIU  Gen  Ed  Goals-  Students: 
a.  develop  habits  of  writing, 
speaking,  and  reasoning  necessary 
for  continued  learning, 
a.i.  communicate  clearly  in  written 
English,  demonstrating  ability  to 
comprehend,  analyze,  and 
interrogate  critically. 

Factual  Knowledge 
Conceptual  Knowledge 
Procedural  Knowledge 
Meta-Cognitive  Knowledge 

5 


H.  Communicate  effectively  orally. 

NIU  Gen  Ed  Goals  - Students: 
a.  develop  habits  of  writing, 
speaking,  and  reasoning 
necessary  for  continued 
learning. 

a.i.  communicate  clearly  in 
written  English,  demonstrating 
ability  to  comprehend,  analyze, 
and  interrogate  critically, 
aii.  communicate  in  a manner 
that  unites  theory,  criticism,  and 
practice  in  speaking  & writing. 

Factual  Knowledge 
Conceptual  Knowledge 
Procedural  Knowledge 
Meta-Cognitive  Knowledge 

I.  Recognize  the  need  for,  and  an 
ability  to  engage  in  life  long  learning 

NIU  Gen  Ed  Goals  - Students: 
a.iv.  are  able  to  access  and  use 
various  information  sources. 
Internet,  text,  and  field  case. 

T. 

9* 

Factual  Knowledge 
Conceptual  Knowledge 
Procedural  Knowledge 
Meta-Cognitive  Knowledge 

J.  Understand  professional,  ethical, 
and  social  responsibilities. 

NIU  Gen  Ed  Goals  - Students: 
d.  develop  social  responsibility  and 
preparation  for  citizenship  through 
global  awareness,  environmental 
sensitivity,  and  an  appreciation  of 
cultural  diversity. 

Factual  Knowledge 
Conceptual  Knowledge 
Procedural  Knowledge 
Meta-Cognitive  Knowledge 

6 


K.  Respect  for  diversity  and  a 
knowledge  of  contemporary 
professional,  societal,  and  global 
issues. 

NIU  Gen  Ed  Goals  - Students: 
d.  develop  social  responsibility  and 
preparation  for  citizenship  through 
global  awareness,  environmental 
sensitivity,  and  an  appreciation  of 
cultural  diversity. 

Factual  Knowledge 
Conceptual  Knowledge 
Procedural  Knowledge 
Meta-Cognitive  Knowledge 

L.  Commitment  to  quality,  timeliness, 
and  continuous  improvement. 

Factual  Knowledge 
Conceptual  Knowledge 
Procedural  Knowledge 
Meta-Cognitive  Knowledge 

M.  Ability  to  program  computers 
and/or  utilize  computer  applications 
effectively. 

Factual  Knowledge 
Conceptual  Knowledge 
Procedural  Knowledge 
Meta-Cognitive  Knowledge 

N.  Ability  to  use  modern  laboratory 
techniques,  skills,  and/or  equipment 
effectively. 

Factual  Knowledge 
Conceptual  Knowledge 
Procedural  Knowledge 
Meta-Cognitive  Knowledge 

7 


O.  Ability  to  manage  projects 
effectively. 

NIU  Gen  Ed  Goals  - Students: 
c.  develop  an  understanding  of 
the  interrelatedness  of  various 
disciplines  by  integrating 
knowledge  from  several 
disciplines  and  applying  that 
knowledge  to  an  understanding 
of  important  problems  and 

Factual  Knowledge 
Conceptual  Knowledge 
Procedural  Knowledge 
Meta-Cognitive  Knowledge 

P.  Ability  to  design,  manipulate,  anc 
manage  industrial  systems. 

NIU  Gen  Ed  Goals  - Students: 
c.  develop  an  understanding  of 
the  interrelatedness  of  various 
disciplines  by  integrating 
knowledge  from  several 
disciplines  and  applying  that 
knowledge  to  an  understanding 
of  important  problems  and 

1 

Factual  Knowledge 
Conceptual  Knowledge 
Procedural  Knowledge 
Meta-Cognitive  Knowledge 

Q.  Ability  to  manage  or  lead 
personnel  effectively. 

Factual  Knowledge 
Conceptual  Knowledge 
Procedural  Knowledge 
Meta-Cognitive  Knowledge 

Addition:  Additional  educational  outcomes  -NIU  General  Education  Goals  - articulated  by  the  overall  program 
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ABET  Engineering  Outcomes 

Bloom's  Knowledge 
Dimension 

Dale’s 

Cone 

Student  Learning 
Objectives 

Bloom's  Cognitive  Process  Dimension 
Knowledge  Comprehension  Application 

Remember  Understand  Apply 

Analysis  Synthesis  Evaluate 

Analyze  Evaluate  Create 

a.  apply  knowledge  of  math. 

Factual  Knowledge: 

science,  engineering 

Conceptual  Knowledge: 
Procedural  Knowledge: 
Meta-cognitive  Knowledge 


b.  design  and  conduct  experiments;  Factual  Knowledge: 
analyze  and  interpret  data 

Conceptual  Knowledge: 
Procedural  Knowledge: 
Meta-cognitive  Knowledge: 


c.  design  a system,  component, 
process  to  meet  desired  needs 
within  realistic  constraints,  e.g. 
economic,  environmental,  social, 
political,  ethical,  health,  safety, 
manufacturability,  & sustainability 


Factual  Knowledge: 
Conceptual  Knowledge: 
Procedural  Knowledge: 
Meta-cognitive  Knowledge: 


d.  function  on  multi-disciplinary  teams  Factual  Knowledge: 

Conceptual  Knowledge: 
Procedural  Knowledge: 
Meta-cognitive  Knowledge: 


1 


ABET  Engineering  Outcomes 


e,  identify,  formulate,  solve  engineering 
problems 


f.  understand  professional  and  ethical 
responsibility 


g.  communicate  effectively 


\ 

h.  understand  impact  of  engineering 
solutions  in  a global  economic, 
environmental,  societal  context 


Bloom's  Knowledge  Dale’s  Student  Learning 

Dimension  Cone  Objectives 


Bloom's  Cognitive  Process  Dimension 

Knowledge  Comprehension  Application 

Remember  Understand  Apply 


Analysis  Synthesis 
Analyze  Evaluate 


Factual  Knowledge: 
Conceptual  Knowledge: 
Procedural  Knowledge 
Conceptual  Knowledge 


Factual  Knowledge: 
Conceptual  Knowledge: 
Procedural  Knowledge: 
Meta-cognitive  Knowledge: 


Factual  Knowledge: 
Conceptual  Knowledge: 
Procedural  Knowledge: 
Meta-cognitive  Knowledge: 

Factual  Knowledge: 
Conceptual  Knowledge: 
Procedural  Knowledge: 
Meta-cognitive  Knowledge: 


Evaluate 

Create 


2 


ABET  Engineering  Outcomes  Bloom's  Knowledge 

Dimension 


Dale’s  Student  Learning 

Cone  Objectives 


Bloom's  Cognitive  Process  Dimension 

Knowledge  Comprehension  Application 

Remember  Understand  Apply 


Analysis  Synthesis 
Analyze  Evaluate 


Evaluate 

Create 


i.  recognize  need  for  and  have  capability  Factual  Knowledge: 
to  engage  in  life  long  learning 

Conceptual  Knowledge: 

Procedural  Knowledge: 

Meta-cognitive  Knowledge: 

j.  knowledge  of  contemporary  issues  Factual  Knowledge: 

Conceptual  Knowledge: 

Procedural  Knowledge: 

Meta-cognitive  Knowledge: 

k.  use  techniques,  skills,  and  modern  Factual  Knowledge: 
engineering  tools  necessary  for 

engineering  practice  Conceptual  Knowledge: 

Procedural  Knowledge: 

Meta-cognitive  Knowledge: 

Addition;  additional  educational  outcomes  articulated  by  the  overall  program  NIU  General  Education  Goals 


3 


Student  Learning 
Objectives 


Procedural  Knowledge; 
Meta-cognitive  Knowledge; 

Factual  Knowledge; 
Conceptual  Knowledge; 

Procedural  Knowledge; 
Meta-cognitive  Knowledge; 


C.  Conduct,  analyze,  and  interpret  Factual  Knowledge; 

experiments;  apply  experimental 

results  to  improve  processes.  Conceptual  Knowledge; 

Gen  Ed  Goals - Students:  Procedural  Knowledge; 

b.  develop  an  ability  to  use  modes 

of  inquiry  across  a variety  of  Meta-cognitive  Knowledge; 

disciplines  in  the  humanities  and 

the  arts,  the  physical  sciences 

and  mathematics,  and  social 

sciences. 

b.iv.  demonstrate  an  ability  to 
use  scientific  methods  and 
theories  to  understand  the 
phenomena  studied  in  the 
natural  and  social  sciences. 


B.  Apply  current  knowledge  and 
adapt  to  emerging  applications 
of  math,  science,  engineering,  and 
technology. 

NIU  Gen  Ed  Goals  - Students: 
a.iii.  perform  basic  computations, 
display  facility  with  use  of  formal 
and  quantitative  reasoning  analysis 
and  problem  solving,  and  interpret 
mathematical  models  and 
statistical  information. 


ABET/TAC/NAIT  Bloom's  Knowledge  Dale  s 

Engineering  & Technology  Dimension  Cone 

A.  Mastery  of  knowledge,  techniques.  Factual  Knowledge; 

skills,  modern  tools  of  disciplines. 

Conceptual  Knowledge; 


1 


Bloom's  Cognitive  Process  Dimension 

Knowledge  Comprehension  Application  Analysis  Synthesis  Evaluate 

Remember  Understand  Apply  Analyze  Evaluate  Create 


ABET/TAC/NAIT  Bloom's  Knowledge 

Engineering  & Technology  Dimension 


Dale’s  Student  Learning 

Cone  Objectives 


Bloom's  Cognitive  Process  Dimension 

Knowledge  Comprehension  Application 

Remember  Understand  Apply 


Analysis  Synthesis 
Analyze  Evaluate 


Evaluate 

Create 


D.  ability  to  apply  creativity  in  the  Factual  Knowledge: 

design  of  systems,  components, 

or  processes  appropriate  to  Conceptual  Knowledge: 

program  objectives. 

Procedural  Knowledge: 


NIU  Gen  Ed  Goals  - Students: 
c.  develop  an  understanding  of 
the  relatedness  of  various 
disciplines  by  integrating 
knowledge  from  several 
disciplines  and  applying  that 
knowledge  to  an  understanding 
of  important  problems  and 

iwi/pv 


Meta-cognitive  Knowledge: 


E.  Eunction  effectively  on  teams.  Factual  Knowledge: 


NIU  Gen  Ed  Goals-Students: 
b.iii.  demonstrate  a knowledge  of 
cultural  traditions  and 
philosophical  ideas  that  have 
shaped  societies,  civilizations,  and 
human  self-conceptions, 
d.  develop  social  responsibility  and 
preparation  for  citizenship  through 
global  awareness,  environmental 
sensitivity,  and  an  appreciation  of 
cultural  diversity. 


Conceptual  Knowledge: 
Procedural  Knowledge 
Meta-cognitive  Knowledge: 


2 


ABET/TAC/NAIT 
Engineering  & Technology 


Bloom's  Knowledge 
Dimension 


Dale’s 

Cone 


Student  Learning 
Objectives 


F.  Identify,  analyze,  and  solve  Factual  Knowledge: 

technical  problems. 

Conceptual  Knowledge: 

Gen  Ed  Goals  - Students:  Procedural  Knowledge: 

b.  develop  an  ability  to  use  modes  of 

inquiry  across  a variety  of  disciplines  Meta-cognitive  Knowledge: 

in  the  humanities  and  the  arts,  the 

physical  sciences  and  mathematics, 

and  social  sciences. 

b.iv.  demonstrate  an  ability  to  use 

scientific  methods  and  theories  to 

understand  the  phenomena  studied  in 

the  natural  and  social  sciences. 


G.  Communicate  effectively  in  writing.  Factual  Knowledge: 

Conceptual  Knowledge: 

NIU  Gen  Ed  Goals-  Students: 

a.  develop  habits  of  writing.  Procedural  Knowledge: 

speaking,  and  reasoning  necessary 

for  continued  learning.  Meta-cognitive  Knowledge: 

a.i.  communicate  clearly  in  written 

English,  demonstrating  ability  to 

comprehend,  analyze,  and 

interrogate  critically. 
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Bloom's  Cognitive  Process  Dimension 

Knowledge  Comprehension  Application  Analysis  Synthesis  Evaluate 

Remember  Understand  Apply  Analyze  Evaluate  Create 


ABET/TAC/NAIT  Bloom's  Knowledge 

Engineering  & Technology  Dimension 


Dale’s  Student  Learning 

Cone  Objectives 


H.  Communicate  effectively  orally. 


NIU  Gen  Ed  Goals  - Students: 
a.  develop  habits  of  writing, 
speaking,  and  reasoning 
necessary  for  continued 
learning. 

a.i.  communicate  clearly  in 
written  English,  demonstrating 
ability  to  comprehend,  analyze, 
and  interrogate  critically, 
aii.  communicate  in  a manner 
that  unites  theory,  criticism,  and 
practice  in  speaking  & writing. 


Factual  Knowledge: 
Conceptual  Knowledge: 
Procedural  Knowledge: 
Meta-cognitive  Knowledge: 


Bloom's  Cognitive  Process  Dimension 

Knowledge  Comprehension  Application 

Remember  Understand  Apply 


Analysis  Synthesis 
Analyze  Evaluate 


Evaluate 

Create 


I.  Recognize  the  need  for,  and  an  Factual  Knowledge: 

ability  to  engage  in  life  long  learning. 

Conceptual  Knowledge: 
Procedural  Knowledge: 
Meta-cognitive  Knowledge: 


Factual  Knowledge: 
Conceptual  Knowledge: 
Procedural  Knowledge: 
Meta-cognitive  Knowledge: 


J.  Understand  professional,  ethical, 
and  social  responsibilities. 


NIU  Gen  Ed  Goals  - Students: 
d.  develop  social  responsibility  and 
preparation  for  citizenship  through 
global  awareness,  environmental 
sensitivity,  and  an  appreciation  of 
cultural  diversity. 


NIU  Gen  Ed  Goals  - Students: 
a.iv.  are  able  to  access  and  use 
various  information  sources. 
Internet,  text,  and  field  case. 
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ABET/TAC/NAIT 
Engineering  & Technology 

K.  Respect  for  diversity  and  a knowledge 
of  contemporary  professional,  societal, 
and  global  issues. 


NIU  Gen  Ed  Goals  - Students: 
d.  develop  social  responsibility  and 
preparation  for  citizenship  through 
global  awareness,  environmental 
sensitivity,  and  an  appreciation  of 
cultural  diversity. 


L.  Commitment  to  quality,  timeliness, 
and  continuous  improvement. 


M.  Ability  to  program  computers 
and/or  utilize  computer  applications 
effectively. 


N.  Ability  to  use  modern  laboratory 
techniques,  skills,  and/or  equipment 
effectively. 


Bloom's  Knowledge 
Dimension 


Dale’s  Student  Learning 

Cone  Objectives 


Bloom's  Cognitive  Process  Dimension 

Knowledge  Comprehension  Application 

Remember  Understand  Apply 


Analysis  Synthesis 
Analyze  Evaluate 


Factual  Knowledge: 
Conceptual  Knowledge: 
Procedural  Knowledge: 
Meta-cognitive  Knowledge: 


Factual  Knowledge: 
Conceptual  Knowledge: 
Procedural  Knowledge: 
Meta-cognitive  Knowledge: 

Factual  Knowledge: 
Conceptual  Knowledge: 
Procedural  Knowledge: 
Meta-Cognitive  Knowledge: 

Factual  Knowledge: 
Conceptual  Knowledge: 
Procedural  Knowledge: 
Meta-Cognitive  Knowledge: 


Evaluate 

Create 
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ABET/TAC/NAIT  Bloom's  Knowledge 

Engineering  & Technology  Dimension 


Dale’s  Student  Learning 

Cone  Objectives 


O.  Ability  to  manage  projects  Factual  Knowledge: 

effectively. 

Conceptual  Knowledge: 


Procedural  Knowledge: 
Meta-cognitive  Knowledge: 


NIU  Gen  Ed  Goals  - Students: 
c.  develop  an  understanding  of 
the  interrelatedness  of  various 
disciplines  by  integrating 
knowledge  from  several 
disciplines  and  applying  that 
knowledge  to  an  understanding 
of  important  problems  and 


Bloom's  Cognitive  Process  Dimension 

Knowledge  Comprehension  Application 

Remember  Understand  Apply 


Analysis  Synthesis 
Analyze  Evaluate 


Evaluate 

Create 


P.  Ability  to  design,  manipulate,  Factual  Knowledge: 

and  manage  industrial  systems. 

Conceptual  Knowledge: 
Procedural  Knowledge: 
Meta-cognitive  Knowledge: 


NIU  Gen  Ed  Goals  - Students: 
c.  develop  an  understanding  of 
the  interrelatedness  of  various 
disciplines  by  integrating 
knowledge  from  several 
disciplines  and  applying  that 
knowledge  to  an  understanding 
of  important  problems  and 


Q.  Ability  to  manage  or  lead  personnel  Factual  Knowledge: 
effectively. 

Conceptual  Knowledge: 

Procedural  Knowledge: 

Meta-cognitive  Knowledge: 

Addition:  additional  educational  outcomes  articulated  by  the  overall  program  **  See  in  text  boxes  above  - NIU  General  Education  Goals 
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Tech  496  Industrial  Project  Management 


Student  Learning  Objectives  and  Outcomes 
Students  will  be  able  to: 

Assessments:  Tests 
Midterm  & Final 

1.  Identify  and  describe  major  problems,  issues,  concerns,  and  solutions  that  relate  to 
projects,  project  management,  project  teams,  and  project  leaders: 

Identify  problems,  issues,  concerns,  and  solutions  (PICS)  that 

a.  occur  during  projects 

b.  relate  to  project  management 

c.  occur  during  team  engagement  on  projects  between  team  members  and/or  team  leaders 

d.  occur  for  team  leaders  of  projects 

e.  are  specific  to  international  projects 

f.  occur  during  projects  that  are  executed  by  a multi-cultural  teams  and  involve  ethnically 
diverse  team  members  working  together 

g.  are  specific  to  team  leaders  of  international  projects  and/or  multicultural  teams  executing 
projects 

2.  Identify  and  describe  best  practices  for  managing  projects  and  leading  teams, 
including  international  projects  and  multi-cultural  teams. 

3.  Perform  effectively  on  a team  to  complete  a technical  project  and  community 
service  project  (multicultural  team  when  possible). 

a.  To  engage  in  the  leadership  of  team  members  or  work  package  groups 

b.  To  engage  in  conflict  resolution  to  resolve  team  issues. 

c.  To  perform  team  and  peer  assessments  throughout  the  project. 

4.  Prepare  the  team  for  project  work  by 

a.  developing  a team  operations  manual 

b.  developing  a peer  and  team  assessment  system 

c.  creating  the  team  organization  and  process 

d.  developing  a team  project  plan 

5.  Exhibit  leadership  while  engaged  in  a team  community  service  project. 

a.  plan 

b.  initiate 

c.  execute 

d.  terminate 

6.  Demonstrate  effective  project 

a.  planning 

b.  initiation 

c.  execution 

d.  evaluation 

e.  termination 

f.  problem  solving 

7.  Demonstrate  effective  use  of  project  management  techniques  and  tools  in  the 
management  of  a technical  project. 

a.  planning 

b.  MS  Project 

c.  finance  procedures 

d.  procurement  procedures 

e.  scheduling 

f.  MACE 

8.  Integrate  mathematics,  the  sciences,  communication,  management,  technical  and 
technological  knowledge  and  skills  to  accomplish  team  and  project  objectives. 

a.  design  a vehicle  to  technical  specifications 

b.  build  the  vehicle  to  technical  specifications 

c.  solve  technical  problems  associated  with  project  design,  construction,  and  evaluation 

d.  test  and  evaluate  the  vehicle  to  technical  specifications 

9.  Design,  develop,  prepare,  and  deliver 

a.  executive  team  presentation 

b.  team  portfolio 

c.  team  website 
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Tech  496  Industrial  Project  Management 


Student  Learning  Objectives  and  Outcomes 

Students  will  be  able  to: 

Corresponding  Assessments: 

1.  Identify  and  describe  major  problems,  issues,  concerns,  and  solutions  that  relate  to 
projects,  project  management,  project  teams,  and  project  leaders: 

Text  Project 

Identify  problems,  issues,  concerns,  and  solutions  (PICS)  that: 
a.  occur  during  projects 

Literature  Study 

b.  relate  to  project  management 

c.  occur  during  team  engagement  on  projects  between  team  members  and/or  team  leaders 

Paper 

d.  occur  for  team  leaders  of  projects 

e.  are  specific  to  international  projects 

Case  Study 

f.  occur  during  projects  that  are  executed  by  a multi-cultural  teams  and  involve  ethnically 
diverse  team  members  working  together 

Career  Project 

g.  are  specific  to  team  leaders  of  international  projects  and/or  multicultural  teams  executing 
projects 

Team  Project 

2.  Identify  and  describe  best  practices  for  managing  projects  and  leading  teams. 

Literature  Study;  Text  Project 

including  international  projects  and  multi-cultural  teams. 

Case  Study;  Paper,  Team  Project 

3.  Perform  effectively  on  a project  team  (hopefully  multicultural  team)  to  complete  a 

Team  Participation  Assessment 

technical  project. 

Team  Assessment 

a.  To  engage  in  the  leadership  of  team  members  or  work  package  groups 

Professor's  Assessment 

b.  To  engage  in  conflict  resolution  to  resolve  team  issues. 

c.  To  perform  team  and  peer  assessments  throughout  the  project 

d.  To  execute  a technical  project 

Team  Project  Outcomes 

4.  Prepare  the  team  for  project  work  by 

Team  Operating  Manual 

a.  developing  a team  operations  manual 

Peer  Assessment  System 

b.  developing  a peer  and  team  assessment  system 

Use  of  Peer  Assessment  System 

c.  creating  the  team  organization  and  process 

d.  developing  an  individual  project  plan 

e.  participating  in  the  development  of  a team  project  plan 

Team  Plan 

5.  Exhibit  leadership  and/or  participation  while  engaged  in  a team  community 

Service  Project  Report  & 

service  project. 

a.  plan 

b.  initiate 

c.  execute 

d.  terminate 

Evaluation 

6.  Demonstrate  effective  project 

Individual  Project  Plan 

a.  planning 

Individual  Portfolio 

b.  initiation 

Peer  Assessment 

c.  execution 

Team  Assessment 

d.  evaluation 

Team  Participation  Assessment 

e.  termination 

Project  Leedback  logs 

f.  problem  solving 

Team  Project  Plan 

g.  leadership 

Team  Project  portfolio, 
presentation,  website 

7.  Demonstrate  effective  use  of  project  management  techniques  and  tools  in  the 

Individual  Project  Plan 

management  of  a technical  project. 

MS  Project  Test 

a.  planning 

Individual  Portfolio 

b.  execution 

Peer  Assessment 

c.  termination 

Team  Assessment 

b.  MS  Project 

Team  Plan 

c.  finance  procedures 

Team  presentation,  portfolio. 

d.  procurement  procedures 

e.  scheduling 

f.  MACE  procedures  and  process 

and  website 
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8.  Integrate  mathematics,  the  sciences,  communication,  management,  technical  and 
technological  knowledge  and  skills  to  accomplish  team  and  project  objectives. 

a.  design  a vehicle  to  technical  specifications 

b.  build  the  vehicle  to  technical  specifications 

c.  solve  technical  problems  associated  with  project  design,  construction,  and  evaluation 

d.  test  and  evaluate  the  vehicle  to  technical  specifications 

Individual  Project  Research 
Individual  Project  Design 
Individual  Portfolio 

9.  Design,  develop,  prepare,  and  deliver 

a.  executive  team  presentation 

b.  team  portfolio 

c.  team  website 

Team  Presentation 
Team  Portfolio 
Team  Website 
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VI.  Course  Topics,  Class  Schedule  & Due  Dates 


Week  & Objectives 

Day  1 Topics/Lab  Activities. 

Due  Dates 

Day  2 Topics/Lab  Activities. 

Due  Dates 

Week  8/28 

Week  9/4 

Week  9/11 

Week  9/18 

Week  9/25 

Week  10/2 

Week  10/9 

Week  10/16  - MIDTERM 

Week  10/23 

Week  10/30 

Week  11/6 

Week  11/13 

Week  11/20 

11/23  THANKSGIVING 

Week  11/27 

Week  12/4 

Week  12/11  FINAL  EXAM 
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VI.  Course  Topics,  Class  Schedule  & Due  Dates 


Week  & Objectives 

Day  1 Topics/Lab  Activities. 

Due  Dates 

Day  2 Topics/Lab  Activities. 

Due  Dates 

Day  3 Topics/Lab  Activities 
Due  Dates 

Week  8/28 

Week  9/4 

Week  9/11 

Week  9/18 

Week  9/25 

Week  10/2 

Week  10/9 

Week  10/16  - MIDTERM 

Week  10/23 

Week  10/30 

Week  11/6 

Week  11/13 

Week  11/20 

11/23  THANKSGIVING 

Week  11/27 

Week  12/4 

Week  12/11  FINAL  XAM 
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Technology  496  - Industrial  Project  Management  - Spring  2004 


Prof:  Dr.  Scarborough  Grad.  Asst: Ph:  753-0210(Dr.  Scarborough)/1570(GA) 

Off.Hrs:  T12-3  Email:  scarboro@ceet.niu.edu 

I.  Catalog  Course  Description:  Industrial  Project  Management  (3).  Basic  concepts,  principles,  and  skills  of  project  management. 
Designed  to  cover  a variety  of  types  of  project  management.  Emphasis  on  computer  tools  and  project  management  techniques.  Analysis  of 
case  studies.  Culminating  project  required. 

II.  Course  Purpose:  To  prepare  project  leaders  and  team  members  to  formally  initiate,  execute  and  terminate  industrial  projects  effectively. 
To  integrate  and  apply  knowledge,  skills,  and  abilities  acquired  or  extended  during  students’  college  careers  (general  education  and  major) 
and  work  experience  to  research,  design,  build  and  finalize  a technical  project  within  a team  environment. 

III.  Required  Text:  Project  Management.  Cleland  & Ireland,  2002,  fourth  edition.  Required:  Datebook/calendar  for  scheduling 
and  notes;  Handout  packet 

IV.  Pre-requisites;  Tech  265-Mfg.  Processes,  Tech  302-  Graphic  Pres.  & Comm.,  Tech  395-Industrial  Data  Processing,  Senior  Status 

Expected  Computer  Usage:  CAD,  MS  Office,  MS  Project,  other,  depending  upon  semester/  team  project.  Required  Laboratory  Team 
Project:  Changes  each  semester;  each  team  will  engage  in  a complex  technical  project  with  specific  technical  standards  to  achieve,  e.g.  Go- 
kart,  3-car  passenger  train,  hovercraft,  paddle  wheel  boat,  personal  transport  vehicle  etc.  Research,  design,  assembly  of  electrical/mechanical 
systems,  testing,  modifications/finalization  with  formal  documentation,  formal  team  products  and  team  requirements.  See  requirements 
section,  handouts,  and  rubrics. 


V.  Student  Learning  Outcomes 

Embedded  NIL  General  Ed 
Goals 

Embedded  NAIT/ABET  Learning 
Standards 

Assessments/Rubrics 

lA/B.  Identify  and  describe  major 
problems,  issnes,  concerns,  and 
solntions  that  relate  to  (a)  projects,  (b) 
project  management,  (c)  project  teams, 
and  (d)  project  leaders,  also  for  (e)  Int’l 
projects  and  (f)  multicultural  (MC) 
teams:  Identify  PICS  - problems, 
issues,  concerns,  and  solutions  for  each. 

2.  Identify  and  describe  best  practices 
for  managing  projects  and  leading 
teams;  include  Int’l  teams  and  MC 
teams. 

a.i.  communicate  clearly  in  English, 
demonstrating  ability  to  comprehend, 
analyze  and  interrogate  critically; 
ii.  communicate  in  a manner  that 
unites  theory,  criticism,  practice  in 
speaking  & listening; 
a.iv.  Aware  of  and  able  to  use 
various  resources,  including  modern 
technology 

g. demonstrate  an  ability  to  communicate 
effectively  in  writing; 

h.  demonstrate  an  ability  to  communicate 
effectively  orally; 

m.  demonstrate  an  ability  to  . . . .utilize 
computer  applications  effectively; 
k.  demonstrates  a respect  for  diversity  and 
knowledge  of  contemporary  professional, 
societal  and  global  issues. 

Text  Project  or  Text  Test 
Research  Literature/Internet;  GP 
Case  study;  group  analysis 
process 

Formal  paper;  group  analysis 
5 minute  learning  papers; 
Individual  portfolios; 

Project  portfolio/website; 
Individual/team  presentations 
Team  participation  & Peer 
Assessment 

Team  Operating  Manual 
Individual  and  Team  Project 
Plan 

3a.  To  demonstrate  effective  project: 

a.  planning, 

b.  initiation, 

c.  execution, 

d.  termination 

e.  evaluation 

f.  problem  solving 

g.  leadership 

3b.  Design,  develop,  and  deliver: 

e.  executive  team  presentation 

f.  team  portfolio 

g.  team  website 

a.i.  communicate  clearly  in  English, 
demonstrating  ability  to  comprehend, 
analyze  and  interrogate  critically; 
ii.  communicate  in  a manner  that 
unites  theory,  criticism,  practice  in 
speaking  & listening; 
a.iii.perform  basic  computations, 
display  facility  with  use  of 
quantitative  reasoning  in  forming 
concepts  for  analysis  and  in  problem 
solving,  and  interpret  mathematical 
models  and  statistical  info 
a.iv.  Aware  of  and  able  to  use 
various  resources,  including  modern 
technology 

a.  demonstrate  appropriate  mastery  of 
knowledge,  techniques,  skills,  and  modern  tools 
of  the  discipline; 

b.  demonstrate  ability  to  apply  current 
knowledge  and  adept  to  emerging  applications 
of  math,  science,  engineering  and  technology; 
d.  demonstrate  ability  to  apply  creativity  in  the 
design  of  systems,  components  or  processes 
appropriate  to  program  objectives; 

f.  demonstrate  ability  to  identify,  analyze,  and 
solve  technical  problems; 
g-h.demonstrate  ability  to  communicate 
effectively  in  writing  and  orally; 

l.  demonstrate  commitment  to  quality, 
timeliness,  and  continuous  improvement; 

m.  demonstrate  ability  to  ....utilize  computer 
applications  effectively; 

0.  demonstrate  an  ability  to  manage  projects, 
industrial  systems,  lead  personnel  effect. 

Individual  project  research 
Individual  project  design 

Written  individual  & team 
plan(s); 

Individual  and  Team 
Portfolio(s);  website(s); 
Individual  and  team 
presentations; 

Industrial  panel  evaluation 
Project  evaluation 
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V.  Student  Learning  Outcomes 

Embedded  NIU  General  Ed 
Goals 

Embedded  NAIT/ABET  Learning 
Standards 

Assessments/Rubrics 

4. T0  demonstrate  effective  use  of 
project  management  techniques  and 
tools  in  the  management  of  a technical 
project: 

a.  planning;  h.  initiation 
c.  execution;  d.  evaluation 
e.  termination;  f.  prohlem- 
solving;  g.  MS  Project; 
h.  finance  procedures;  i. 

procurement  procedures; 
j.  scheduling;  k.  MACE 
process  & procedures 

5.  To  integrate  mathematics,  the 
sciences,  English,  management, 
technical  and  technological  knowledge 
and  skills  to  accomplish  team  and 
project  objectives: 

(a)  Design,  (b)  Build  a vehicle  to 
technical  specification  that  will  run;  (c) 
Solve  technical  problems  encountered; 
(d)  test  and  evaluate  the  vehicle  to 
technical  specifications 

a.  cultivate  habits  of  writing, 
speaking,  quantitative  reasoning  for 
continued  learning: 

a. iii.perform  basic  computations, 
display  facility  with  use  of 
quantitative  reasoning  in  forming 
concepts  for  analysis  & in  problem 
solving,  and  interpret  mathematical 
models  & statistical  information; 
iv.  Aware  of  & able  to  use  various 
resources,  including  technology; 

b.  develop  an  ability  to  use  modes  of 
inquiry  across  a variety  of  disciplines 
in  the  physical  sciences, 
mathematics: 

iii.  demonstrate  ability  to  use 
scientific  methods,  theories  to 
science  phenomena; 

c.  develops  understanding  of 
discipline  interrelatedness,  applying 
that  knowledge  to  an  understanding 
of  important  problems  & issues. 

a.  demonstrate  an  appropriate  mastery  of  the 
knowledge,  techniques,  skills,  and  modern  tools 
of  the  discipline; 

b.  demonstrate  ability  to  apply  current 
knowledge,  adapt  to  emerging  applications  of 
math,  science,  engineering  and  technology; 

d.  demonstrate  an  ability  to  apply  creativity  in 
the  design  of  systems,  components  or  processes 
appropriate  to  program  objectives; 
f.  demonstrate  an  ability  to  identify,  analyze, 
and  solve  technical  problems; 
g-h.demonstrate  an  ability  to  communicate 
effectively  in  writing  and  orally; 

l.  demonstrate  a commitment  to  quality, 
timeliness,  and  continuous  improvement; 

m.  demonstrate  an  ability  to  . . . .utilize  computer 
applications  effectively; 

0.  demonstrate  an  ability  to  manage  projects; 

p.  demonstrate  an  ability  to  manage  and 
manipulate  industrial  systems; 

q.  demonstrate  knowledge,  strategies  and/or 
techniques  of  how  to  lead  personnel  and  teams 
effectively 

Project  plan 

Project  execution  & completion 
to  technical  standards 
Project  termination  with  lessons 
learned 

Project  evaluation  by  industrial 
panel 

Project  presentation 
Project  portfolio  and  website 
MS  Project  2003  test  and 
application  in  project  planning, 
execution,  termination, 
assessment  and  evaluation 
MACE-Project  assessment 
(Plan  compliance  & adjustments) 
Logs 

Project  Design 

Project  building  using  technical 
Processes 

Project  testing  procedures 
Testing  data  collection 
Testing  evaluation  process 
Project  Metric  process 

6.  Prepare  the  team  for  project  and 
team  work  by: 

a.  developing  a team  operations  manual 

b.  developing  peer  and  team  assessment 
system 

c.  creating  team  organization  & process 

d.  developing  team  project  plan 

7.  Demonstrate  effective  team 
performance  (hopefully  MC  team) 
while: 

a.  engaged  in  a community  service 
project;  plan  execute,  report  relevance. 

b.  engaged  in  the  initiation,  planning, 
execution  and  termination  of  a technical 
project 

c.  engaged  in  course  L.  activities 

d.  develops  social  responsibility  & 
preparation  for  citizenship  through 
service  and  an  appreciation  of 
cultural  diversity. 

e.  demonstrate  ability  to  function  effectively  on 
teams; 

j.  demonstrate  ability  to  understand  profess- 
ional, ethical,  social  responsibilities; 

k.  demonstrate  respect  for  diversity,  knowledge 
of  contemporary  professional,  societal  and 
global  issues; 

Team  Operations  Manual; 
Team  Plan 

Team  presentation;  portfolios; 
website; 

Team  peer,,  team,  & conflict 
assessments/logs; 

Industrial  panel  evaluation; 
Formal  paper  ;5  minute  learning 
papers 

Team  success  rubric 
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VI.  Topics,  Class  Schedule  & Due  Dates 


Week/Date 

Topics 

Date 

Topics/Lab  Act. 

Assignment  Due  Dates 

1 

Course 

Intro 

9:30  Writing  Center  Requirements 
(Jacky) 

9:  45Career  Project  Intro  and  Requirements 
(Norwood) 

10:00  Team  Selection/Scheduling 

Teaming 

Team 

Assess. 

Team  Skills  Bank 
Finalize  Teams  & Schedules 

Plan  Team  Service  Project 

Due  1/20 

Writing  Center  Appointments 
Project  Research 
Bring  Planner 

10:30  Course  Intro 
1 1 :30  Legacy  Group 

Project  Research  Review 

Commnnity  Service  Art.  & 
Plan  Dnel/20  4:00pm 

Use  of  Planner  & The  Nature  of  Multitasking 

Project  Research  Assignment 

Schedule  Writing  Center 
NOW!!!! 

2 

Text  1-4, 19 

TEXT  Highlights 

Teaming 

Project  Design  Lab 

Writing  Center  Appts.  Due  1/25 
Text  Proj.  1-4, 19,  20  due  1/25 
Industry  Case  ID  due  1/25 

*210  sched. 

Final  Project  Research  due  1/27 

3 

Teaming 
Text  18  & HB 

Project  Teams:  hidden  agendas,  teamwork, 
effective  teams  & members,  member  roles  & 
responsibilities  - Peer  Assessment 

(Team  Packet  Required) 

Research 
Design 
*210  sched. 

Project  Design  Lab 

Text  Project  18,  21  due  2/1 
Project  Design  due  2/4  Friday 

4 

Teaming 
Text  20 

Project  Teams:  conflict  resolution,  decision- 
making, teams  in  trouble,  empowerment,  trust, 
recognition 

(Team  Packet  Required) 

Teaming 

Team  Manual  Lab 

Lit.  Research  Table  due  2/8 
Career  Project  due  2/11 

5 

Project 
Planning 
Text  11,6 

Project  Planning  - Section  I Rubric  & TEXT 
Vision,  Mission,  Intro,  Purpose,  Scope, 
Objectives,  Deliverables,  Charter,  Org.  Charts, 
Stakeholder  Analysis,  Com  Interface,  Project 
Review,  Change  Plan  [Paper  due] 

Teaming 

Research,  Case,  Paper,  Career  Validation 
Activity  - Group  Process 

Team  Manual  Lab 

Industry  Case  due  2/15 
Text  Proj.  6,  8, 11, 16  due  2/15 
Team  Manual  due  2/14 

6 

Project 
Planning 
Text  13, 

Project  Planning  - Section  II  Rubric  & TEXT 
Business  & Proj.  Success  Eactors,  SWOT 
Analysis,  Project  Constraints,  Risk  Analysis, 
Contingency  Plans  & Trade  Offs,  Statement  of 
Work,  Goals,  Work  Break-down  Structure 

Project 

Planning 

*210  sched. 

Project  Planning  - Section  II  Lab 

Community  Leadership  Project 
and  Articles  due  2/25  Friday 

Paper  due  2/21 

7 

Project 

Planning 

Section  II  Rubric  & TEXT  (Continued) 

Life  Cycle,  Productivity  Plan, 

Quality  Standards  & Metrics, 

Project  Monitoring,  Assessment,  Control  and 
Evaluation,  Linear  Charts,  Resource 
Plan/Budget  MS  Project  - PM  Software 

Software 

Workshop 

*210  sched. 

MS  Project-PM  Software 

Text  Proj.  5,9, 12, 13, 14, 15 
due  3/1 

8 

Project 

Planning 

Section  III  Rubric  & TEXT 
Environmental/Safety  Plan,  Security  Plan, 
Documentation/Configuration  Mgmt.  Plan, 
Project  Divestment  & Termination  Plan 

Software 
Workshop 
*210  sched. 

MS  Project-PM  software 

Individual  Plans  due  3/11 
Software  Test  due  by  3/9 

BREAK 

3/17 

BREAK 

9 

Project  Development  & Teamwork 

3/24 

Project  Development  & Teamwork  [Logs] 

Individual  Portfolios  due  3/22 
Team  Plans  due  3/25 

10 

Project  Development  & Teamwork 

3/31 

Project  Development  & Teamwork  [Logs] 

Individual  Portfolios  due 

11 

Project  Development  & Teamwork 

4/7 

Project  Development  & Teamwork[Logs] 

12 

Project  Development  & Teamwork 

4/14 

Project  Development  & Teamwork[Logs] 

13 

Project  Development  & Teamwork 

4/21 

Project  Development  & Teamwork  [Logs] 

14 

Project  Development  & Teamwork 

4/28 

Project  Testing  and  Initial  Assessment 

Proj.  Test. /Assess  due  4/28-29 

15 

[Final  Project  Assessment  & Grade] 

[Peer  Assessments  Executed  & Due] 
[Team  Member  Participation  Determined] 

5/5 

[Team  Presentations  8:30am-12:30] 
[Team  Portfolio/Website/Success  due] 

Team  ProjectAssessment  due 
5/3 

Team  Pres./Port./Web.  Due  5/5 

16  Mav 

Finals 

Week 

5/12 

Final  Exam:  TBD 

If  needed  to  confirm  competencies 

VII.  Course  Requirements; 


Individual  Course  Requirements:  Points: 

Team  Course  Requirements:  Points: 

Grading: 

Text  Project  (broken  into  sections  for  due  dates)  7 

Project  Research 

5 

Project  Design 

5 

Benchmark=98- 100 

Literature/Internet  Research  Tables  A & B 

7 

Team  Manual 

5 

A=93-100 

(Projs/Tms/Lead/Infl  Tms/MCTms/MCLd) 

Community/Leadership  Service  Project  /Arts 

3 

B=85-92 

Career  Project 

5 

Team  Project  Plan 

5 

C=77-84 

Ind.  Case  Study 

5 

Team  Project  & Assessment  (Final  Exam) 

7 

D=70-76 

Paper 

7 

*Peer  Assessment  Process  /Team  Success 

5 

F=  Below  70 

Midterm:  Individual  Project  Plan 

7 

*Team  Member  Participation  (Ind.  Pts.3) 

Software  Workshop/Test 

5 

Team  Presentation  & Success  (Pinal  Exam) 

5 

*Team  Participation  Awarded  by  Team 

3 

(Ind.  Pres.  Impacts  Profs.  Points) 

Project  Feedback  Logs 

1 

Team  Project  Portfolio  & Website 

5 

Individual  Portfolio  & Assessment  Process 

3 

(Final  Exam) 

Pinal  II:  TBD  if  needed  to  confirm  competencies. 

Professors  Overall  Assessment  & Ind.  Pres. 

5 

Total  Individual  Points  Possible 

65 

Total  Team  Points  Possible 

35 

Professor’s  Privilege 

See  Note  #2 

VIII.  Cheating: 

Cheating  is  unacceptable;  refer  to  the  NIU  Judicial  Code;  any  students  cheating  will  be  dismissed  from  the  course  immediately. 

IX.  Professor’s  Role:  This  course  involves  the  professor  and  graduate  assistant  in  a variety  of  roles;  the  professor  will  provide  a scenario, 
objectives,  and  standards  and  then  guide,  coach,  and  direct  most  of  the  time,  however,  there  will  be  some  lectures.  This  course  is 
performance  based,  thus,  there  are  usually  no  traditional  objective  tests.  There  are  subjective  tests  in  the  form  of  the  5-10  minute  learning 
papers,  essays  and  the  text  project  to  determine  concept  attainment.  Students  will  construct  knowledge/skills  while  engaged  in  learning  & 
performances.  Assessment  will  occur  ^ learning  occurs. 

X.  Professor’s  Notes: 

1.  Unexcused  absences  could  result  in  one  lettergrade  reduction  each  (7pts).  Class/lab/  team  meetings/work  sessions  attendance  mandatory. 
Tardiness  unacceptable.  Door  may  close  when  class  begins;  late  admittance  may  not  be  permitted  according  to  prof.’s  prerogative. 
Unexcused  class/team  tardies,  1 point  per  30  minutes  IF  you  are  allowed  in  and  door  is  open;  don’t  count  on  door  being  open. 

2.  The  professor  reserves  the  right  to  determine  the  final  grade  in  the  case  of  a student  who  does  not  perform  on  the  team. 

3.  Unexcused  late  projects/assignments  will  result  in  point  reduction,  2 points  per  day  late. 

4.  Dress  code:  no  hats  in  lab  ever!  Professional  dress  required  for  final  presentation. 

5.  Monitor  language  in  class/lab  at  all  times;  good  grammar  and  communication  skills  expected  at  all  times;  professional  language  expected. 

6.  Students  are  required  to  see  the  Writing  Center  tutor  for  all  written  assignments  until  approved  otherwise,  at  least  2 visits  per  assignment; 
3 visits  required  for  paper.  (1)  Meet  once  to  design  paper,  then  meet  with  draft  in  hand  (2-3)  twice  and  rewrite.  An  appointment  to  plan  the 
written  assignment  with  no  draft  for  review  would  still  require  2 other  visits  for  all  other  assignments. 

7.  Unannounced  individual  portfolio  checks  throughout  course;  5 point  penalties  for  portfolios  not  up  to  date  each  time. 

8.  No  cell  phone  ringers  in  the  class  or  lab  at  any  time;  5 points  deducted  for  in-class  interruptions.  See  professor  exception  approval. 
9.Students  can  not  pass  class  without  ALL  assignments  turned  in.  Student  will  receive  an  I (incomplete)  until  all  assignments  are 
turned  in.  Penalties  may  occur  for  grades  of  Incomplete. 

XI.  Support  Services  Available  for  Students:  The  NIU  writing  center  provides  tutoring  for  writing.  Students  in  this  class  are  required  to 
use  that  service  for  all  written  assignments;  each  writing  assignment  requires  two  visits/critiques  and  rewrites  before  assignment  can  be 
handed  in  to  professor.  Tutor  signatures  and  forms  are  required  to  be  turned  in  with  written  products.  Math  and  science  tutors  available  in 
College.  NIU  accommodations  for  any  student  with  special  needs.  See  professor  individually. 

XII.  References  on  reference  in  Founders  Library  on  NIU  main  campus:  Kerzner.  Smith.  Project  Management  & Teamwork.  McGrawHill. 
Angus,  Gundersen,  Cultinane.  Planning.  Performing  and  Control-  ling  Projects.  Prentice  Hall.  2000;  Dinsmore.  Human  Factors  in  Project 
Management.  Dinsmore.  Project  Management.  Thomsett.  The  Little  Black  Book  of  Project  Management.  AMACOM;  Kerzner,  Thamhain. 

Projet  Management  Operating  Guidelines.  VNR.;  Rosenau.  Successful  Project  Management.  VNR.;  Weiss,  Wysocki.  5 -Phase  Project 
Management.  Addison  Wesley;  Cleland,  Gareis.  Global  Project  Management  Handbook.  McGrawHill.;  Miller.  Visual  Project  Planning  & 
Scheduling;  Barkley,  Saylor.  Customer  Driven  Project  Management.  McGrawHill;  Lewis.  Mastering  Project  Management.  McGrawHill; 
Forseberg,  Mooz,  Goterman.  Visualizing  Project  Management.  Wiley;  Dinsmore.  Winning  in  Businesss  With  Enterprise  Project  Management. 
AMACOM;  Graham,  Englund.  Creating  an  Environment  for  Successfu  Projects.  JoseyBass;Gray,  Larson.  Project  Management.  McGraw  Hill. 
2000.;  Cleland.  Project  Management.  McGrawHill;  Meredith,  Mantel.  Project  Management.  Wiley.  2000;  Lewis.  Team-based  Project 
Management.  AMACOM;  Kliem,  Ludin.  Project  Management  Practitioner’s  Handbook.  AMACOM;  Kerzner.  In  Search  of  Excellence  in 
Project  Management.  ITP  VNR;  Ruskin,  Estes.  What  Every  Engineer  Should  Know  About  Project  Management.  Dekker;  Buttrick.  The  Project 
Workout.  ET  Pitman;  Rowenau.  Project  Management  for  Engineers.  Ran  Nostrand  Reinhold;  Cleland.  Eield  Guide  to  Project  Management. 
ITP  VNR;  Briner,  Geddings,  Hastings.  Project  Leadership.  Van  Nostrand  Reinhold;  Rosenau.  Successful  Project  Mgmt.  Wi. 


4 


Logs:  Periodically  you  will  be  asked  to  complete  a log  about  how  you  feel  the  team  and  project  are  progressing.  These  must  be  completed  and  turned  in. 
Individual  Component  of  Team  Presentation;  Speaking,  non-verbal  communication,  presentation  skills,  content,  grammar/wording  visuals,  style, 
organization,  use  of  technology,  humor,  etc.  graded  individually  during  team  presentation.  Remember  that  each  team  member  must  demonstrate  speaking 
and  presentation  skills.  Teams  could  acquire  the  full  point  value,  but  individuals  will  be  assessed  on  their  individual  performance  as  well.  Professional  dress 
required.  See  Presentation  Outline/Rubric. 

Team  Participation  Points  Awarded  by  Team:  Each  team  member  will  be  allocated  points  for  team  participation.  Teams  will  award  points  to  team 
members  for  quality  of  work  and  participation.  Points  will  serve  to  “grade”  participation.  Dr.  Scarborough  validates  that  the  distribution  is  appropriate  for 
participation  observed.  Full  participation  is  expected  of  each  team  member.  Tardiness  or  absences  from  team  meetings,  class,  labs  are  not  acceptable 
behaviors.  You  will  be  asked  to  explain  to  the  class  openly  why  you  are  late  or  absent  and  points  will  be  deducted.  See  Rubric.  Individual/  Group  Process. 
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Employment 

Career  Project:  a)  Interview  Mr.  Norwood,  the  CEET  Career  Planning  & Placement  specialist,  on  the  assigned  topic;  engage  in  group  process. 
Document  findings  as  assigned  (TBD);  b)Research  jobs/positions/career  in  project  management;  Bring  in  copies  of  10  position 
announcements  which  review  expectations,  required  knowledge,  skills,  background  for  those  seeking  to  become  project  managers,  team 
leaders,  or  project  team  members;  c)  Design  and  develop  a resume  to  use  to  seek  such  a position,  but  also  make  it  applicable  for  other  industrial 
technology,  management,  engineering,  etc.  positions.  Have  it  reviewed  and  approved  by  Mr.  Norwood  for  inclusion  into  personal  496  portfolio.  Mr. 
Norwood  will  grade  this  project. 

Individual  Portfolio:  This  portfolio  has  a somewhat  different  focus.  Although  it  may  contain  everything  in  the  team  portfolio  for  job-seeking 
purposes,  it  must  also  include  all  individual  work,  including  Writing  Center  Reviews  and  multiple  iterations  of  particular  products.  Use  Course 
Requirements  list  on  Course  Syllabus  (above)  and  Team  Portfolio  Rubric  to  determine  what  is  to  be  included.  You  will  participate  in  assessment 
activities  throughout  the  semester,  including  analysis  and  reflections  about  what  your  strengths  and  weaknesses  are  and  what  you  can  do  to  improve  or 
continue  well.  The  portfolio  must  be  professionally  presented,  e.g.  typed  tabs,  etc.  Final  Reflections  at  end  of  semester/questions  to  answer.. 

Team  Requirements: 

Community  Service  Project:  Each  team  has  to  research,  determine,  plan  and  execute  an  8 hour  service  project.  Research  one  article  per 
team  member  on  the  benefits  of  community  service  and  leadership  by  local  industrial  personnel.  Generate  a brief  team  plan  of  what,  who, 
when  and  where.  It  should  include  a goal,  operational  objectives,  expected  outcomes  and  benefit  to  group  served.  Prepare  an  informal 
presentation  about  what  you  learned,  how  you  felt  and  your  potential  future  in  community  service.  See  Rubric.  Individual/  Group  Process. 

Team  Process 

Team  Manual:  The  team  manual  includes  all  team  operational  policies  and  procedures,  the  team  problem-solving  process, 
communication  strategy  and  procedures,  decision-making  process,  authority  linear  charts,  team  roles  and  responsibilities,  etc.  The  team  is 
to  provide  evidence  that  it  operated  using  the  team  manual  as  its  structure,  process  and  guiding  document.  See  Outline/Rubric.  Group 
Process. 

Included  in  the  Team  Manual  are  the  following  critical  components,  plus  others:  See  Team  Manual  Outline/Rubric.  Group  Process. 

Team  Skills  Bank;  Each  team  will  prepare  a team  skills  bank  that  identifies  all  individual  talent,  skills,  knowledge  that  each  team 
member  brings  to  the  project.  This  bank  will  be  used  to  organize  the  team,  project,  work  packages  and  deliverables.  Group 
Process. 

Team/Project  Charter,  Logo,  Company  & Project  Organizational  Chart:  Each  team  will  create  an  official  charter,  identify  a 
team  logo,  and  design  an  organizational  chart  for  their  company  and  their  team/project.  Group  Process. 

Team  Assessment  Inventory(ies):  Each  team  will  design  and  develop  a peer  and  team  status  inventory  to  use  to  monitor  team 
process;  they  will  also  adapt  a conflict  management  inventory  to  use  to  monitor  the  team  conflict  resolution  process.  The 
information  gained  from  using  these  inventories  will  be  used  to  build  and  strengthen  the  team  and  to  identify  and  solve  team 
issues  or  problems.  Growth  and  development  should  be  an  outcome  of  using  these  instruments.  Each  team  must  produce  a report 
of  results  from  using  these  instruments  and  assessment  process  twice  during  the  project  period. 


Team  Project  Plan:  Each  team  must  write/develop  a team  plan;  however,  the  team  may  not  begin  on  the  team  plan  until  all  individual  plans 
are  graded  and  returned.  The  plan  outline  is  the  same  as  the  individual  plan.  Each  team  must  produce  a plan  for  the  technical  project 
assigned  and  use  the  plan  as  a compliance  document  to  monitor,  assess,  control  and  evaluate  the  project.  See  Outline/Rubric.  Group  Proc. 

Final  Exam 

Team  Project:  Each  team  will  be  responsible  for  designing  and  developing  a technical  project.  You  will  generate  technical  standards  to 
achieve  and  the  metrics  to  use  to  measure  the  standards  achievement  level.  The  project  must  “function”  or  “work”  to  be  accepted  for  a 
grade.  It  must  meet  the  standards  at  the  level  described  in  the  team  plan  using  the  metrics  predetermined.  Every  team  member  must  have 
major  project  role  and  responsibilities.  The  team  must  complete  the  project  by  the  deadline  on  the  syllabus.  The  project  is  the  “vehicle” 
providing  evidence  of  high  performance  teaming  and  project  management  as  well  as  the  knowledge,  skills,  and  abilities  from  academic 
career  and  work  experience.  Team  derived/Professor  approved-  predetermined  -Standards/metrics  = grading  Rubric.  Group  Process. 

Team  Portfolio:  The  portfolio  is  the  culminating  documentation  of  all  project  and  team  work.  It  must  include  information  on  every  topic 
listed  in  the  outline/rubric.  It  should  include  pictures,  mechanical  drawings,  etc.  and  be  professionally  produced  in  hard-copy  form.  An 
operator’s  and  maintenance  manual  must  be  developed  and  included  for  the  technical  project  (product).  See  Rubric.  Group  Process. 

Team  Website:  Each  team  is  to  design  and  produce  a team  web-site  which  will  serve  as  an  electronic  portfolio.  This  website/  portfolio 
must  be  presented  during  the  team  presentation.  The  outline  is  the  same  as  the  hard-copy  portfolio.  A CD  must  be  included  in  the  hard 
copy  of  the  team  portfolio.  See  Rubric.  Group  Process. 

Final  Team  Presentation:  Each  team  is  to  professionally  present  their  project,  portfolio/website  and  information  for  each  category  on  the 
presentation  outline.  This  is  a formal  presentation  where  communication  skills,  presentation  skills,  etc.  will  be  graded.  Professional  dress 
required.  An  industrial  panel  will  observe  the  presentations.  Presentation  CD  must  be  in  Portfolio.  See  Outline/Rubric. 


6 


Course  Requirements  Check  Off 


Individual  Contributions: 

(7)  Text  Project 

(5)  Project  Research 

(5)  Project  Design 


Benchmark=98-100  (This  means  that  you  set  the  standard  for  others.) 

A=93-100  points 
B=92.9-85  points 
C=84.9-77  points 
D=76.9-70  points 
F=Below  70  points 


(7)  Literature/Internet  Research  Tables  A & B 


(5)  Career  Project 


(5)  Industrial  Case  Study 
(7)  Paper 

(7)  Midterm:  Individual  Project  Plan 

(5)  Software  Workshop/Test 

(3)  *Team  Participation  Awarded  by  Team  Members 
(confirmed  by  Professor) 

( 1 ) Project  Feedback  Logs 


Note:  To  keep  track  of  your  progress, 
add  the  possible  points  of  work  to  date; 
then  figure  the  percentage,  e.g. 

Text  (7)  + P. Research  (5)  + P.Design  (5)=  17 
.93  X 17  = 15.81  = lowest  possible  score  or 
point  value  to  maintain  an  A (lowest  A). 

IF  your  goal  is  to  be  a Benchmark  Student, 
where  your  work  best  exemplifies  the  course's 
highest  standards. . .where  you  set  the  standard, 
then  you  must  maintain  no  lower  than  98%  or 
ultimately  98  points  for  the  course. 


(3)  Individual  Portfolio  & Assessment  Process  If  a team's  goal  is  to  be  a Benchmark  Team, 

where  the  team  best  exemplifies  the  course's 

(5)  Professor’s  Overall  Assessment  highest  standards  for  teams. . .where  the  team 

sets  the  standard  for  other  teams,  then  every 

(65)  Total  Individual  Points  Possible  team  member  in  that  team  must  maintain 

98%  or  ultimately  98  points  each  for  the 
Team  Contributions:  course. 


(5)  Team  Manual 

(3)  Community/Leadership  Service  Project/ Articles 
(5)  Team  Project  Plan 

(7)  Team  Project  & Assessment  (Final  Exam) 

(5)  *Peer  Assessment  Process/Team  Success 

(5)  *Team  Member  Participation 

(5)  Team  Presentation  & Success  (Final  Exam) 

*Individual  Presentation  (in  Team  Pinal  Presentation) 
(5)  Team  Project  Portfolio  & Website 

(35)  Total  Team  Points  Possible 
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Technology  496  - Industrial  Project  Management 

Prof:  Dr.  Scarborough  Grad.  Asst: Ph:  753-0210(Dr.  Scarborough)/1570(GA)  Off.Hrs: 

T12-3  Email: 

I.  Catalog  Course  Description:  Industrial  Project  Management  (3).  Basic  concepts,  principles,  and  skills 
of  project  management.  Designed  to  cover  a variety  of  types  of  project  management.  Emphasis  on  computer 
tools  and  project  management  techniques.  Analysis  of  case  studies.  Culminating  project  required. 

II.  Course  Purpose  & Objectives:  To  prepare  project  leaders  and  team  members  to  formally  initiate, 
execute  and  terminate  industrial  projects  effectively.  To  integrate  and  apply  knowledge,  skills,  and  abilities 
acquired  or  extended  during  students’  college  careers  (general  education  and  major)  and  work  experience  to 
research,  design,  build  and  finalize  a technical  project  within  a team  and  formal  project  environment. 

III.  Required  Text:  Project  Management.  Cleland  & Ireland,  2006  or  latest  edition.  Required:  Date 
book/Calendar  for  scheduling  and  notes;  Handout  packet. 

IV.  Pre-requisites:  Tech  265-Mfg.  Processes;  Tech  302-  Graphic  Pres.&  Comm.;  Tech  395-Ind.  Data 
Processing;  Senior  Status 

Expected  Computer  Usage:  CAD,  MS  Office,  MS  Project,  CNC,  industrial  equipment,  or  other,  depending 
upon  semester/  team  project.  Required  Laboratory  Team  Project:  Changes  each  semester;  each  team  will 
engage  in  a complex  technical  project  with  specific  technical  standards  to  achieve,  e.g.  Go-kart,  3-car 
passenger  train,  hovercraft,  paddle  wheel  boat,  personal  transport  vehicle  etc.  Research,  design,  assembly  of 
electrical/mechanical  systems,  testing,  modifications/finalization  with  formal  documentation,  formal  team 
products  and  team  requirements.  See  requirements  section,  handouts,  and  rubrics. 


V.  Course  Requirements: 


Individual  Course  Requirements: 

Points: 

Team  Course  Requirements:  Points: 

Grading: 

Text  Project  (broken  into  sections/due  dates) 

7 

Team  Operations  Manual 

5 

Benchmark 

Project  Research 

5 

Community  Leadership  Service 

3 

A=98-100 

Project  Design 

5 

Project  & Articles 

Literature/Internet  Research  Tables  A & B 

7 

Team  Project  Plan 

5 

A=93-97 

(Proj  s/T  ms/Lead/Int  ’ 1/MCT  ms/MCLd) 

Team  Project  & Assessment 

7 

B=92-85 

Career  Project 

5 

(Final  Exam) 

C=  84-77 

Individual  Case  Study 

5 

*Peer  Assessment  Process 

5 

D=76-70 

Paper 

7 

& Team  Success  Assessment 

p=69-below 

Midterm:  Individual  Project  Plan 

7 

*Team  Member  Participation 

Software  Workshop/Test 

5 

(Ind.  Pts.  3/5) 

*Team  Participation  Awarded  by  Team 

3 

Team  Presentation  & Success 

5 

Project  Feedback  Logs 

1 

(Pinal  Exam) 

Individual  Portfolio  & Assessment  Process 

3 

Team  Project  Portfolio  & Website 

5 

(Final  Exam) 

Professor’s  Overall  Assessment 

Pinal  Exam  II:  TBD 

Ind.Presentation  within  Team  Presentation. 

5 

(if  needed  to  confirm  competencies) 

Total  Individual  Points  Possible 

65 

Total  Team  Points  Possible 

35 

Professor’s  Privilege 

See  Note  #2 

1 


VI.  Student  Learning  Outcomes 


Student  Learning  Outcomes 

Embedded  NIU  General  Ed 
Goals 

Embedded  NAIT/ABET 
Learning  Standards 

Assessments/Rubrics 

lA/B.  Identify  and  describe  major 
problems,  issnes,  concerns,  and 
solntions  (PICS)  that  relate  to  (a) 
projects,  (b)  project  management,  (c) 
project  teams,  and  (d)  project  leaders, 
also  for  (e)  Int’l  projects  and  (f) 
multicultural  (MC)  teams. 

2.  Identify  and  describe  best  practices 
for  managing  projects  and  leading 
teams;  include  Int’l  teams  and  MC 
teams. 

a.i.  communicate  clearly  in  English, 
demonstrating  ability  to  comprehend, 
analyze  and  interrogate  critically; 
ii.  communicate  in  a manner  that 
unites  theory,  criticism,  practice  in 
speaking  & listening; 
a.iv.  Aware  of  and  able  to  use 
various  resources,  including  modern 
technology 

g. demonstrate  an  ability  to  communicate 
effectively  in  writing; 

h.  demonstrate  an  ability  to  communicate 
effectively  orally; 

m.  demonstrate  an  ability  to  . . . .utilize 
computer  applications  effectively; 
k.  demonstrates  a respect  for  diversity  and 
knowledge  of  contemporary  professional, 
societal  and  global  issues. 

Text  Project  or  Text  Test 
Research-  Literature/Internet; 
Case  study; 

Group  analysis  process 
Formal  paper;  group  analysis 
1-5  minute  learning  papers; 
Individual  portfolios; 

Team  Project  portfolio/website; 
Individual/team  presentations 
Team  participation  & Peer 
Assessment 

Team  Operating  Manual 
Individual  and  Team  Project  Plan 
Community  leadership  project 

3a.  Demonstrate  effective  project 
management  of  a technical  project  using 
appropriate  PM  techniques,  tools,  and 
processes: 

a.  planning, 

b.  initiation, 

c.  execution, 

d.  termination 

e.  evaluation 

f.  problem  solving 

g.  leadership 

h.  financial  management 

i.  procurement  management 

j.  scheduling 

k.  MACE  process  and  procedures 

3b.  Design,  develop,  and  deliver: 

e.  executive  team  presentation 

f.  team  portfolio 

g.  team  website 

4.  To  integrate  mathematics,  the 
sciences,  English,  management, 
technical,  technological  systems 
knowledge  and  skills  to  accomplish 
individual  and  team  project  objectives: 
(a)  Design,  (b)  Build  a vehicle  to 
technical  specification  that  will  operate; 
(c)  Solve  technical  problems 
encountered;  (d)  test  and  evaluate  the 
vehicle  for  meeting  technical 
specifications  and  standards 

a.  cultivate  habits  of  writing, 
speaking,  quantitative  reasoning  for 
continued  learning: 

a.i.  communicate  clearly  in  English, 
demonstrating  ability  to  comprehend, 
analyze  and  interrogate  critically; 
ii.  communicate  in  a manner  that 
unites  theory,  criticism,  practice  in 
speaking  & listening; 
a.iii.perform  basic  computations, 
display  facility  with  use  of 
quantitative  reasoning  in  forming 
concepts  for  analysis  and  in  problem 
solving,  and  interpret  mathematical 
models  and  statistical  info 

a. iv.  Aware  of  and  able  to  use 
various  resources,  including  modern 
technology 

b.  develop  an  ability  to  use  modes  of 
inquiry  across  a variety  of  disciplines 
in  the  physical  sciences, 
mathematics: 

b. iii.  demonstrate  ability  to  use 
scientific  methods,  theories  to 
science  phenomena; 

c.  develops  understanding  of 
discipline  interrelatedness,  applying 
that  knowledge  to  an  understanding 
of  important  problems  & issues. 

a.  demonstrate  appropriate  mastery  of 
knowledge,  techniques,  skills,  and  modern 
tools  of  the  discipline; 

b.  demonstrate  ability  to  apply  current 
knowledge  and  adept  to  emerging 
applications  of  math,  science,  engineering 
and  technology; 

d.  demonstrate  ability  to  apply  creativity  in 
the  design  of  systems,  components  or 
processes  appropriate  to  program 
objectives; 

f.  demonstrate  ability  to  identify,  analyze, 
and  solve  technical  problems; 
g-h.  demonstrate  ability  to  communicate 
effectively  in  writing  and  orally; 

l.  demonstrate  commitment  to  quality, 
timeliness,  and  continuous  improvement; 

m.  demonstrate  ability  to  ....utilize 
computer  applications  effectively; 

0.  demonstrate  an  ability  to  manage 
projects,  industrial  systems,  lead  personnel 
effect. 

p.  demonstrate  an  ability  to  manage  and 
manipulate  industrial  systems; 

q.  demonstrate  knowledge,  strategies  and/or 
techniques  of  how  to  lead  personnel  and 
teams  effectively 

Individual  & Team  project  research 
Individual  & Team  project  design 

Written  individual  & team  plan(s); 

Technical  project  prototype  product 
produced  to  technical  standards  and 
specifications  using  technical 
processes 

Project  testing  & evaluation  against 
established  standards  and 
specifications  using  formal 
evaluation  tools  and  procedures 

MS  Project  2003  test  and 
application  in  project  planning, 
execution,  termination,  assessment 
and  evaluation 
MACE-Project  assessment 
(Plan  compliance  & adjustments) 
Individual  & Team  Logs 

Individual  and  Team  Portfolio(s); 
website(s); 

Individual  and  team  presentations; 
Industrial  panel  evaluation 

Project  termination  with  lessons 
learned 

Project  evaluation  by  industrial  panel 

5.  Develop  the  team  for  project  and 
team  work  by: 

a.  developing  a team  operations  manual 

b.  developing  peer  and  team  assessment 
system 

c.  creating  team  organization  & process 

d.  developing  team  project  plan 

6.  Demonstrate  effective  team 
performance  (hopefully  MC  team) 
while: 

a.  engaged  in  a community  service 
project;  plan,  execute,  & report 
relevance. 

b.  engaged  in  the  initiation,  planning, 
execution  and  termination  of  a technical 
project 

c.  engaged  in  course  Team  & Project 

d.  develops  social  responsibility  & 
preparation  for  citizenship  through 
service  and  an  appreciation  of 
cultural  diversity. 

e.  demonstrate  ability  to  function 
effectively  on  teams; 

j.  demonstrate  ability  to  understand 
profess-ional,  ethical,  social  responsibilities; 

k.  demonstrate  respect  for  diversity, 
knowledge  of  contemporary  professional, 
societal  and  global  issues; 

Team  Operations  Manual; 
Team  Plan 

Team  presentation;  portfolios; 
website; 

Team  peer,,  team,  & conflict 
assessments/logs; 

Industrial  panel  evaluation; 
Formal  paper  ;5  minute  learning 
papers 

Team  success  rubric 

activities 


VII.  Topics,  Class  Schedule  & Due  Dates 


Week/Date 

Topics 

Date 

Topics/Lab  Act. 

Assignment  Due  Dates 

1 

Course 

Intro 

9:30  Writing  Center  Requirements 
(Jacky) 

9:  45Career  Project  Intro  and  Requirements 
(Norwood) 

10:00  Team  Selection/Scheduling 

10:30  Course  Intro 
1 1 :30  Legacy  Group 

Use  of  Planner  & The  Nature  of  Multitasking 
Project  Research  Assignment 

Schedule  Writing  Center  NOW!!!! 

Teaming 

Team 

Assess. 

Team  Skills  Bank 
Finalize  Teams  & Schedules 

Plan  Team  Service  Project 

Project  Research  Review 

Due  1/20 

Writing  Center  Appointments 
Project  Research 
Bring  Planner 

Community  Service  Art.  & 
Plan  Duel/20  4:00pm 

2 

Text  1-4, 19 

TEXT  Highlights 

Teaming 
*210  sched. 

Project  Design  Lab 

Writing  Center  Appts.  Due  1/25 
Text  ProJ.  1-4, 19,  20  due  1/25 
Industry  Case  ID  due  1/25 

Final  Project  Research  due  1/27 

3 

Teaming 
Text  18  & HB 

Project  Teams:  hidden  agendas,  teamwork, 
effective  teams  & members,  member  roles  & 
responsibilities  - Peer  Assessment 

(Team  Packet  Required) 

Research 
Design 
*210  sched. 

Project  Design  Lab 

Text  Project  18,  21  due  2/1 
Project  Design  due  2/4  Friday 

4 

Teaming 
Text  20 

Project  Teams:  conflict  resolution,  decision- 
making, teams  in  trouble,  empowerment,  trust, 
recognition 

(Team  Packet  Required) 

Teaming 

Team  Manual  Lab 

Lit.  Research  Table  due  2/8 
Career  Project  due  2/11 

5 

Project 
Planning 
Text  11,6 

Project  Planning  - Section  I Rubric  & TEXT 
Vision,  Mission,  Intro,  Purpose,  Scope, 
Objectives,  Deliverables,  Charter,  Org.  Charts, 
Stakeholder  Analysis,  Com  Interface,  Project 
Review,  Change  Plan  [Paper  due] 

Teaming 

Research,  Case,  Paper,  Career  Validation 
Activity  - Group  Process 

Team  Manual  Lab 

Industry  Case  due  2/15 
Text  Proj.  6,  8, 11, 16  due  2/15 
Team  Manual  due  2/14 

6 

Project 
Planning 
Text  13, 

Project  Planning  - Section  II  Rubric  & TEXT 
Business  & Proj.  Success  Eactors,  SWOT 
Analysis,  Project  Constraints,  Risk  Analysis, 
Contingency  Plans  & Trade  Offs,  Statement  of 
Work,  Goals,  Work  Break-down  Structure 

Projeet 

Planning 

*210  sched. 

Project  Planning  - Section  II  Lab 

Community  Leadership  Project 
and  Articles  due  2/25  Friday 

Paper  due  2/21 

7 

Project 

Planning 

Section  II  Rubric  & TEXT  (Continued) 

Life  Cycle,  Productivity  Plan, 

Quality  Standards  & Metrics, 

Project  Monitoring,  Assessment,  Control  and 
Evaluation,  Linear  Charts,  Resource 
Plan/Budget  MS  Project  - PM  Software 

Software 

Workshop 

*210  sched. 

MS  Project-PM  Software 

Text  Proj.  5,9, 12, 13, 14, 15 
due  3/1 

8 

Project 

Planning 

Section  III  Rubric  & TEXT 
Environmental/Safety  Plan,  Security  Plan, 
Documentation/Configuration  Mgmt.  Plan, 
Project  Divestment  & Termination  Plan 

Software 
Workshop 
*210  sched. 

MS  Project-PM  software 

Individual  Plans  due  3/11 
Software  Test  due  by  3/9 

BREAK 

3/17 

BREAK 

9 

Project  Development  & Teamwork 

3/24 

Project  Development  & Teamwork  [Logs] 

Individual  Portfolios  due  3/22 
Team  Plans  due  3/25 

10 

Project  Development  & Teamwork 

3/31 

Project  Development  & Teamwork  [Logs] 

Individual  Portfolios  due 

11 

Project  Development  & Teamwork 

4/7 

Project  Development  & Teamwork[Logs] 

12 

Project  Development  & Teamwork 

4/14 

Project  Development  & Teamwork[Logs] 

13 

Project  Development  & Teamwork 

4/21 

Project  Development  & Teamwork  [Logs] 

14 

Project  Development  & Teamwork 

4/28 

Project  Testing  and  Initial  Assessment 

Proj.  Test. /Assess  due  4/28-29 

15 

[Final  Project  Assessment  & Grade] 

[Peer  Assessments  Executed  & Due] 
[Team  Member  Participation  Determined] 

5/5 

[Team  Presentations  8:30am-12:30] 
[Team  Portfolio/Website/Success  due] 

Team  ProJectAssessment  due 
5/3 

Team  Pres./Port./Web.  Due  5/5 

16  Mav 

Finals 

Week 

5/12 

Final  Exam:  TBD 

If  needed  to  confirm  competencies 

3 


VIII.  Course  Requirements  Explanation  --  Individual  Requirements 


Technical  Research  and  Design 

Project  Research:  Research  project  assigned.  More  information  about  this  research  will  be  provided  in  class.  However,  it  will  entail  an 
Internet/Literature  search,  possibly  interviewing  technical  experts,  local  or  suburban  vendors  or  manufacturers,  or  other  professors,  and/or  researching 
specific  technicalities.  It  will  also  include  research  of  all  properties  of  materials,  mathematics,  and  scientific  principles,  theories  involved  in  the 
technical  aspects  of  the  project.  Use  research  information  to  design  the  project.  See  Rubric.  Individual  and  Group  Process. 

Project  Design:  Students  will  design  and  prepare  visuals  and  working  drawings,  schematics,  etc.  for  the  project  using  prior  design  and  computer 
aided  drafting  or  mechanical  drawing  knowledge  and  skills.  See  Rubric.  Individual  and  group  process. 


Real  World  Validation  - Culminating  Paper 

Literature/Internet  Research  A:  Search  the  literature  (Internet)  on  project  management,  project  teams,  and  project  leadership;  identify  45  quality 
sources,  15  each  about  (a)industrial  projects, (h)  project  teams,  and  (c)project  leadership.  Develop  a literature/source  review  Table  summarizing 
what  the  literature/sources  revealed.  Topics  of  focus  should  be  the(l.)problems,  issues,  concerns,  (PICs)  difficulties  that  arise  on  projects  or  for  the 
teams  and  leaders  and  (2.)success  strategies  that  have  worked  for  projects,  project  teams  or  leaders  in  resolving  the  problems/issues.  There  must  be 
45  sources;  these  must  be  from  major  recognized  journals  or  books  on  the  topics.  You  may,  however,  include  up  to  five  non-traditional  sources,  e.g. 
Internet  sources  from  industrial  groups,  project  teams,  etc.  Sources  must  show  depth  in  content;  short  “briefs”  are  not  acceptable.  Copy  all  sources  if 
not  books  on  diskette  or  CD  rather  than  hardcopies.  See  Table  Format  and  Rubric.  Group  Process-Be  prepared  to  discuss;  thus,  if  no 
hardcopies  available  for  reference,  you  need  to  know  the  material  well.  **Create  tables  that  are  categorized,  numbered  and  reveal  in-depth 
information  with  solutions.  May  use  for  your  paper. 

Literature  A + B = Total  Table  (See  Rubric) 

Literature/Internet  Research  B:  Also,  research  (a)international  projects,  (b)multicultural  teams,  and  (c)international  project  leadership  with  a 
multicultural  team;  identify  15  (5  for  each  topic)  Internet  and/or  literature  sources  that  discuss  (l.)problems,  issues  and  (2.)  best  practices,  benefits, 
successes  of  multicultural/international  projects,  teams,  and  project  or  team  leadership.  Summarize  the  information  learned  by  organizing  it  into  a 
Table  identifying  the  source  author,  title,  main  points  on  problems,  issues,  and  benefits  and  your  comments.  See  Rubric.  Individual/Group  Process- 
Be  prepared  to  discuss;  thus,  your  if  no  hardcopies  available  for  your  reference,  you  need  to  know  the  material  well.  **Create  tables  that  are 
categorized,  numbered  and  reveal  in-depth  information  with  solutions.  May  use  for  paper. 


Industrial  Case  Study:  Identify  a company  that  will  allow  you  to  visit  and  interview  an  industrial  project  team.  Interview  a project  leader  or 
manager  and  at  least  three  project  team  members  or  2 project  leaders  and  2 project  team  members.  (l.)Ask  them  to  identify  all  problems,  issues, 
concerns,  (PICs)  or  difficulties  encountered  on  the  project,  about  the  project,  team.,  and  project  leadership.  Have  them  explain  in  detail;  (2.)then,  also 
ask  them  what  strategies  are  successful  for  projects,  teams,  and  project  leaders.  Create  a table  of  questions  and  responses  and  present  what  was 
learned  as  “real-time”  research.  See  Rubric  and  Format.  Individual  and  Group  Process.  Incorporate  the  results  into  your  paper. 

Formal  Paper:  Meet  with  WC  tutor  to  orgauize  paper.  Develop  a paper  about  projects,  teams  and  project  leadership;  develop  the  issues 
and  solutions  in  greater  depth;  draw  conclusions  and  describe  effective  project  management,  effective  project  teams,  and  effective  project  leadership. 
What  strategies,  techniques,  processes  should  be  used  to  have  a more  successful  project,  team,  or  leader/leadership  process?  End  with  very  specific 
recommendations  to  guide  your  project  team  on  each  of  the  3 primary  topics.  Then  include  a section  on  how  international  projects  and  multi-cultural 
teams  differ,  what  additional  concerns,  problems,  and  issues  occur  when  operating  internationally  with  diverse  cultures.  Make  recommendations  for 
successful  international  projects  and  on  how  to  be  a more  effective  leader  of  multicultural  teams.  Sixty  (60)  sources  required  (45  -l-  15).  These  60 
sources  may  or  may  not  be  the  same  ones  that  you  identified  for  the  literature  review  table.  **Incorporate  the  results  of  industrial  case 
study  into  your  paper  as  well.  Use  the  APA  writing  style  manual.  Identify  all  sources  in  the  paper’s  text  and  in  References  Cited  using  the  APA 
style  format.  Writing  skills  are  seriously  graded  on  this  product.  See  Writing  Rubric,  Paper  Outline  & Rubric.  Individual  and  Group  Process. 


Project  Planning  - Midterm  Exam 

MS  Project  Software  Workshops/Test:  Participate  in  the  software  workshop(s).  Complete  Test.  MS  Project  documents  required  in  PLANs. 
Midterm  - Individual  Project  Plan:  Use  the  outline  & rubric  provided  as  a guide,  develop  a detailed  project  plan.  The  plan  will  not  be  accepted 
unless  every  category  is  complete.  Reference  the  text,  other  sources  in  the  library  or  through  the  Internet,  or  sources  listed  on  the  course  syllabus.  All 
members  of  a team  must  have  their  plan  in  and  graded  before  they  will  be  approved  to  work  on  the  “team”  plan.  This  is  another  product  where 
writing  will  be  graded  seriously.  This  is  technical  writing  which  is  different  than  the  narrative  or  prose  approach  used  in  the  above  assignments.  See 
Plan  Outline/Rubric.  I/GP 


Team  Requirements: 

Community  Service  Project:  Each  team  has  to  research,  determine,  plan  and  execute  an  8 hour  service  project.  Research  one  article 
per  team  member  on  the  benefits  of  community  service  and  leadership  by  local  industrial  personnel.  Generate  a brief  team  plan  of 
what,  who,  when  and  where.  It  should  include  a goal,  operational  objectives,  expected  outcomes  and  benefit  to  group  served.  Prepare 
an  informal  presentation  about  what  you  learned,  how  you  felt  and  your  potential  future  in  community  service.  See  Rubric. 
Individual/  Group  Process 

Team  Process 

Team  Manual:  The  team  manual  includes  all  team  operational  policies  and  procedures,  the  team  prohlem-solving  process,  communication  strategy 
and  procedures,  decision-making  process,  authority  linear  charts,  team  roles  and  responsihilities,  etc.  The  team  is  to  provide  evidence  that  it  operated 
using  the  team  manual  as  its  stmcture,  process  and  guiding  document.  See  Outline/Rubric.  Group  Process. 

Included  in  the  Team  Manual  are  the  following  critical  components,  plus  others:  See  Team  Manual  Outline/Rubric.  Group  Process. 

Team  Skills  Bank:  Each  team  will  prepare  a team  skills  bank  that  identifies  all  individual  talent,  skills,  knowledge  that  each  team  member 
brings  to  the  project.  This  bank  will  be  used  to  organize  the  team,  project,  work  packages  and  deliverables.  Group  Process. 

Team/Project  Charter,  Logo,  Company  & Project  Organizational  Chart:  Each  team  will  create  an  official  charter,  identify  a team  logo, 
and  design  an  organizational  chart  for  their  company  and  their  team/project.  Group  Process. 

Team  Assessment  Inventory(ies):  Each  team  will  design  and  develop  a peer  and  team  status  inventory  to  use  to  monitor  team  process;  they 
will  also  adapt  a conflict  management  inventory  to  use  to  monitor  the  team  conflict  resolution  process.  The  information  gained  from  using 
these  inventories  will  be  used  to  build  and  strengthen  the  team  and  to  identify  and  solve  team  issues  or  problems.  Growth  and  development 
should  be  an  outcome  of  using  these  instruments.  Each  team  must  produce  a report  of  results  from  using  these  instruments  and  assessment 
process  twice  during  the  project  period. 

Team  Project  Plan:  Each  team  must  write/develop  a team  plan;  however,  the  team  may  not  begin  on  the  team  plan  until  all  individual 
plans  are  graded  and  returned.  The  plan  outline  is  the  same  as  the  individual  plan.  Each  team  must  produce  a plan  for  the  technical 
project  assigned  and  use  the  plan  as  a compliance  document  to  monitor,  assess,  control  and  evaluate  the  project.  See  Outline/Rubric. 
Group  Proc. 

Logs:  Periodically  you  will  be  asked  to  complete  a log  about  how  you  feel  the  team  and  project  are  progressing.  Completed  and  turn  in. 

Individual  Component  of  Team  Presentation;  Speaking,  non-verbal  communication,  presentation  skills,  content,  grammar/wording  visuals, 
style,  organization,  use  of  technology,  humor,  etc.  graded  individually  during  team  presentation.  Remember  that  each  team  member  must  demonstrate 
speaking  and  presentation  skills.  Teams  could  acquire  the  full  point  value,  but  individuals  will  be  assessed  on  their  individual  performance  as  well. 
Professional  dress  required.  See  Presentation  Outline/Rubric. 

Team  Participation  Points  Awarded  by  Team:  Each  team  member  will  be  allocated  points  for  team  participation.  Teams  will  award  points  to 
team  members  for  quality  of  work  and  participation.  Points  will  serve  to  “grade”  participation.  Dr.  Scarborough  validates  that  the  distribution  is 
appropriate  for  participation  observed.  Full  participation  is  expected  of  each  team  member.  Tardiness  or  absences  from  team  meetings,  class,  labs  are 
not  acceptable  behaviors.  You  will  be  asked  to  explain  to  the  class  openly  why  you  are  late  or  absent  and  points  will  be  deducted.  See  Rubric. 
Individual/  Group  Process 
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Final  Exam 

Team  Project:  Each  team  will  be  responsible  for  designing  and  developing  a technical  project.  You  will  generate  technical  standards  to 
achieve  and  the  metrics  to  use  to  measure  the  standards  achievement  level.  The  project  must  “function”  or  “work”  to  be  accepted  for  a grade.  It 
must  meet  the  standards  at  the  level  described  in  the  team  plan  using  the  metrics  predetermined.  Every  team  member  must  have  major  project 
role  and  responsibilities.  The  team  must  complete  the  project  by  the  deadline  on  the  syllabus.  The  project  is  the  “vehicle”  providing  evidence  of 
high  performance  teaming  and  project  management  as  well  as  the  knowledge,  skills,  and  abilities  from  academic  career  and  work  experience. 
Team  derived/Professor  approved-  predetermined  -Standards/metrics  = grading  Rubric.  Group  Process. 

Team  Portfolio:  The  portfolio  is  the  culminating  documentation  of  all  project  and  team  work.  It  must  include  information  on  every  topic  listed 
in  the  outline/rubric.  It  should  include  pictures,  mechanical  drawings,  etc.  and  be  professionally  produced  in  hard-copy  form.  An  operator’s  and 
maintenance  manual  must  be  developed  and  included  for  the  technical  project  (product).  See  Rubric.  Group  Process. 

Team  Website:  Each  team  is  to  design  and  produce  a team  web-site  which  will  serve  as  an  electronic  portfolio.  This  website/  portfolio  must  be 
presented  during  the  team  presentation.  The  outline  is  the  same  as  the  hard-copy  portfolio.  A CD  must  be  included  in  the  hard  copy  of  the  team 
portfolio.  See  Rubric.  Group  Process. 

Final  Team  Presentation:  Each  team  is  to  professionally  present  their  project,  portfolio/website  and  information  for  each 
category  on  the  presentation  outline.  This  is  a formal  presentation  where  communication  skills,  presentation  skills,  etc.  will  be 
graded.  Professional  dress  required.  An  industrial  panel  will  observe  the  presentations.  Presentation  CD  must  be  in  Portfolio. 

See  Outline/Rubric. 


VIII.  Cheating: 

Cheating  is  unacceptable;  refer  to  the  NIU  Judicial  Code;  any  students  cheating  will  be  dismissed  from  the  course  immediately. 

IX.  Academic  Misconduct:  Refer  to  the  NIU  Judicial  Code;  Immediate  and  appropriate  actions  will  occur  for  any  students  behaving 
inappropriately,  e.g.  cheating,  will  be  dismissed  from  the  course  immediately. 

X.  Professor’s  Role:  This  course  involves  the  professor  and  graduate  assistant  in  a variety  of  roles;  the  professor  will  provide  a scenario, 
objectives,  and  standards  and  then  guide,  coach,  and  direct  most  of  the  time,  however,  there  will  be  some  lectures.  This  course  is  performance 
based,  thus,  there  are  usually  no  traditional  objective  tests.  There  are  subjective  tests  in  the  form  of  the  5-10  minute  learning  papers,  essays  and 
the  text  project  to  determine  concept  attainment.  Students  will  construct  knowledge/skills  while  engaged  in  learning  & performances. 

Assessment  will  occur  as  learning  occurs. 

XI.  Professor’s  Notes: 

1.  Unexcused  absences  could  result  in  one  letter  grade  reduction  each  (7pts).  Class/lab/  team  meetings/work  sessions  attendance  mandatory. 
Tardiness  unacceptable.  Door  may  close  when  class  begins;  late  admittance  may  not  be  permitted  according  to  prof.’s  prerogative. 
Unexcused  class/team  tardies,  I point  per  30  minutes  IF  you  are  allowed  in  and  door  is  open;  don’t  count  on  door  being  open. 

2.  The  professor  reserves  the  right  to  determine  the  final  grade  in  the  case  of  a student  who  does  not  perform  on  the  team. 

3.  Unexcused  late  projects/assignments  will  result  in  point  reduction,  2 points  per  day  late. 

4.  Dress  code:  no  hats  in  lab  ever!  Professional  dress  required  for  final  presentation. 

5.  Monitor  language  in  class/lab  at  all  times;  good  grammar  and  communication  skills  expected  at  all  times;  professional  language  expected. 

6.  Students  are  required  to  see  the  Writing  Center  tutor  for  all  written  assignments  until  approved  otherwise,  at  least  2 visits  per  assignment;  3 
visits  required  for  paper.  (I)  Meet  once  to  design  paper,  then  meet  with  draft  in  hand  (2-3)  twice  and  rewrite.  An  appointment  to  plan  the 
written  assignment  with  no  draft  for  review  would  still  require  2 other  visits  for  all  other  assignments. 

7.  Unannounced  individual  portfolio  checks  throughout  course;  5 point  penalties  for  portfolios  not  up  to  date  each  time. 

8.  No  cell  phone  ringers  in  the  class  or  lab  at  any  time;  5 points  deducted  for  in-class  interruptions.  See  professor  exception  approval. 
9.Students  can  not  pass  class  without  AUU  assignments  turned  in.  Student  will  receive  an  I (incomplete)  until  all  assignments  are  turned 
in.  Penalties  may  occur  for  grades  of  Incomplete. 

XII.  Support  Services  Available  for  Students:  The  NIU  writing  center  provides  tutoring  for  writing.  Students  in  this  class  are  required  to  use 

that  service  for  all  written  assignments;  each  writing  assignment  requires  two  visits/critiques  and  rewrites  before  assignment  can  be  handed  in  to 
professor.  Tutor  signatures  and  forms  are  required  to  be  turned  in  with  written  products.  Math  and  science  tutors  available  in  College.  NIU 
accommodations  for  any  student  with  special  needs.  See  professor  individually. 

XIII.  References  on  reference  in  Founders  Uibrary  on  NIU  main  campus:  Kerzner.  Smith.  Project  Mana2ement  & Teamwork. 
McGrawHill.  Angus,  Gundersen,  Cultinane.  Planning.  Performing  and  Control-  ling  Projects.  Prentice  Hall.  2000;  Dinsmore.  Human 
Factors  in  Project  Management.  Dinsmore.  Project  Management.  Thomsett.  The  Uittle  Black  Book  of  Project  Management. 
AMACOM;  Kerzner,  Thamhain.  Project  Management  Operating  Guidelines.  VNR.;  Rosenau.  Successful  Project  Management.  VNR.; 
Weiss,  Wysocki.  5-Phase  Project  Management.  Addison  Wesley;  Cleland,  Gareis.  Global  Project  Management  Handbook. 
McGrawHill.;  Miller.  Visual  Project  Planning  & Scheduling;  Barkley,  Saylor.  Customer  Driven  Project  Management.  McGrawHill; 
Uewis.  Mastering  Project  Management.  McGrawHill;  Forseberg,  Mooz,  Goterman.  Visualizing  Project  Management.  Wiley;  Dinsmore. 
Winning  in  Business  With  Enterprise  Project  Management.  AMACOM;  Graham,  Englund.  Creating  an  Environment  for  Successful 
Projects.  JoseyBass;  Gray,  Uarson.  Project  Management.  McGraw  Hill.  2000.^  Cleland.  Project  Management.  McGrawHill;  Meredith, 
Mantel.  Project  Management.  Wiley.  2000;  Uewis.  Team-based  Project  Management.  AMACOM;  Kliem,  Eudin.  Project 
Management  Practitioner’s  Handbook.  AMACOM;  Kerzner.  In  Search  of  Excellence  in  Project  Management.  ITP  VNR;  Ruskin, 
Estes.  What  Every  Engineer  Should  Know  About  Project  Management.  Dekker;  Buttrick.  The  Project  Workout.  FT  Pitman; 
Rowenau.  Project  Management  for  Engineers.  Ran  Nostrand  Reinhold;  Cleland.  Field  Guide  to  Project  Management.  ITP  VNR; 
Briner,  Geddings,  Hastings.  Project  Eeadership.  Van  Nostrand  Reinhold;  Rosenau.  Successful  Project  Mgmt.  Wi. 
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XIV.  Course  Requirements  Check  Off 

Individual  Contributions:  Benchmark=98-I00  (This  means  that  you  set  the  standard  for  others.) 

(7)  Text  Project  A=93-100  points 

B=92.9-85  points 

(5)  Project  Research  C=84.9-77  points 

D=76.9-70  points 

(5)  Project  Design  F=Below  70  points 

(7)  Literature/Internet  Research  Tables  A & B 

(5)  Career  Project 

(5)  Industrial  Case  Study 

(7)  Paper 

(7)  Midterm:  Individual  Project  Plan 

A). 

(5)  Software  Workshop/Test 

(1)  Project  Feedback  Logs 

(5)  Professor’s  Overall  Assessment 

(65)  Total  Individual  Points  Possible 

Team  Contributions: 

(5)  Team  Manual 

(3)  Community/Leadership  Service  Project/ Articles 

(5)  Team  Project  Plan 

(7)  Team  Project  & Assessment  (Final  Exam) 

(5)  *Peer  Assessment  Process/Team  Success 

(5)  *Team  Member  Participation 

(5)  Team  Presentation  & Success  (Final  Exam) 

*Individual  Presentation  (in  Team  Pinal  Presentation) 

(5)  Team  Project  Portfolio  & Website 

(35)  Total  Team  Points  Possible 


IP  a teams  goal  is  to  be  a Benchmark 
Team,  where  the  team  best  exemplifies 
the  course's  highest  standards  for 
teams . . . where  the  team  sets  the  standard 
for  other  teams,  then  every  team 
Member  in  that  team  must  maintain 
98%  or  ultimately  98  points  each  for  the 
course. 


Note:  To  keep  track  of  your  progress, 
add  the  possible  points  of  work  to  date; 
then  figure  the  percentage,  e.g. 

Text  (7)+P.Research(5)+P.Design(5)=17 
.93  X 17  = 15.81  = lowest  possible  score 
or  point  value  to  maintain  an  A(lowest 

IP  your  goal  is  to  be  a Benchmark 
Student,  where  your  work  best 
exemplifies  the(confirmed  by  Professor) 
course's  highest  standards  ...where  you 
set  the  standard,  then  you  must  maintain 
no  lower  than  98%  or  ultimately  98 
points  for  the  course. 
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Course  Title  and  Number 


Student  Learning  Objectives  and  Outcomes 

Assessments:  Test  Alignments 
Midterm  & Final 

Student  Learning  Objectives/Outcomes- 
Major 

Student  Learning  Objectives 
- Minor 

Corresponding  Tests  and 
Test  Items 

1 

a 

b 

c 

d 

2 

a 

b 

c 

d 

3 

a 

b 

c 

d 

4 

a 

b 

c 

d 

5 

a 

b 

c 

d 

6 

a 

b 

c 

d 

7 

a 

b 

c 

d 

8 

a 

b 

c 

d 

9 

a 

b 

c 

d 

10 

a 

b 

c 

d 

11 

12 

1 


Scarborough,  2006;  Technology  496  (based  on  Kuhs  et  al.,  2001) 


Scarborough,  2006;  Technology  496  (based  on  Kuhs  et  al.,  2001) 


Scarborough,  2006  - Tech  496  -(based  upon  Kuhs  et  al.,  2001,  p.157) 


Analysis  Table  for  Assessments  - Bloom’s  Taxonomy-  Cognitive  Dimension 

Assessments 

Knowledge 

Comprehension 

Application 

Analyze 

Synthesize 

Evaluate 

Remember 

Understand 

Apply 

Analyze 

Evaluate 

Create 

Pre- Assessment 

Midterm 

Quizes  1,2,3 

Final 

Performance 

Taskl 

Performance 
Task  2 

Performance 
Task  3 

Performance 
Task  3 

Research 

Homework 

Lab  or  Field 
Experiments 

Miscellaneous: 

Individual 

Assessments 

Web  Page 

Group/Team 

Assessments 

Group 

Discussion 

Round  Table 
Discussion 

Oral 

Presentation 

Visual 

Presentation 

Technical  Project 

Case  Study 

1 
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The  Learning  Environment:  Modeis  of  Teaching 
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Scarborough,  2007  (Teaching  Modeis  by  Joyce  et  ai.,  2004) 
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The  Learning  Environment:  Models  of  Teaching 
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The  Learning  Environment:  Models  of  Teaching 


Student  Learning  Communities 


fnductiveX  /Concept' 
Sjhinkinq/  Attainmer 


Scientific 

inquiry 


(i\ferenc6 


mformationN 


Processing 
Models 


’ictureN 

Word 

iductivjC 
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Teaching 


Personal 

Family 


onceptuai 

eveiopme 
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Behavioral 
Systems 
Family 


Student  Teams  of  Expert  Learners 


^^Tech  496-Scarborough,  2007  (Teaching  Models  by  Joyce  et  ai.,  2004) 


Foundation:  Concepts  that  Apply  to  Learning 


Concepts  - Chapter  1,  p.  3 

Description 

Meaning  for  Me  and  My 
Practice 

Changes  I will  make  based 
upon  my  understanding  of  this 
concept? 

Where  will  these  changes  “show  up” 
in  the  teaching  and  learning 
experiences  throughout  the  semester? 

Constructivism,  p.  12 

Metacognition,  p.  14 

Scaffolding,  p.  14 

Zone  of  Proximal  Development,  p.  16 

(optimal  mismatches  with  tasks  given  to  students) 
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I.  Models  of  Teaching  - Information  Processing  Models 


Models 

Description 

Strengths 

Inductive  thinking,  Ch.  3,  p.  41 

(classification-oriented) 

Concept  Attainment,  Ch.  4,  p.  59 
(includes  concept  formation) 

The  Picture-Word  Inductive  Model 
Ch.  5,  p.  77 

Scientific  Inquiry,  Ch.  6,  p.  101 
Inquiry  Training 

Mnemonics,  Ch.7,  p.l31 

(memory  assists) 

Synectics,  Ch.  8,  p.  155 
(includes  metaphoric  activity) 

Advance  Organizers,  Ch.  9,  p.  187 


II.  Models  of  Teaching  - Social  Models 


Model 

Description 

Strengths 

Partners  in  Learning,  Ch.  10,  p.  205 

Positive  Interdependence,  p.  21 1 

Structured  Inquiry,  p.  221 

Group  Investigation  p.  213, 14-227 

Role  Playing,  Ch.ll,  p.  229 

Jurisprudential  Inquiry,  Ch.  11,  p.  249 


Weaknesses 


How  I can  use  this  model-describe 


III.  Models  of  Teaching  - Personal  Family 


Model 

Description 

Strengths 

Weaknesses 

How  I can  use  this  model-descrihe 

Nondirective  teaching,  ch.  12, 
p.  271 

Enhancing  Self-esteem,  ch.  13, 
p.  283 

Conceptual  Development 
Ch.  13,  p.  290 
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IV.  Models  of  Teaching  - Behavioral  Models 


Model 

Description 

Strengths 

Mastery  Learning,  ch.  14,  p.  303 
Programmed  Schedule,  p.  310 
Programmed  Schedule,  3.11 
(task  performance 
reinforcement) 

Direct  Instruction,  Ch.  15,  p.  313 

Simulation,  Ch.  16,  p.  323 
Training  and  Self-Training 

Social  Learning,  Ch.l4 
(includes  training  & self- 
training) 


Bloom’s  Learning  Taxonomy  - Analysis  Chart  for  Assessments 

Assessment Knowledge Comprehension 

Remember Understand 

Midterm 


Application 

Apply 


Analyze 

Analyze 


Synthesize 

Evaluate 


Evaluate 

Create 


Final 


Performance 


Performance 


Performance 


VI.  Course  Topics,  Class  Schedule  & Due  Dates 


Topics/Lab  Activities  & Due  Dates 

TM 

TS 

LS 

Topics/Lab  Activities  & Due  Dates 

VI-  Course  Topics,  Class  Schedule  & Due  Dates  

Week  TM  LS  Topics/Lab  Activities  -pjyj  LS  Topics/Lab  Activities  TM  LS  Topics/Lab  Activities 
Obi  TS  Due  Dates  Due  Dates  TS  Due  Dates 

TS 
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Teaching  Learning  Calendar/Content  Analysis  Legend 


Teaching  Models 
(Joyce,  Weil,  Calhoun,  2004) 

IT  - inductive  thinking 

CA  - concept  attainment 

PWIM  - picture  word  induction  model 

Scl  - scientific  inquiry 

M - mneumonics 

S - synectics 

AO  - advance  organizers 
Partners 

CL-I  - cooperative  learning-informal 

CL-F  - cooperative  learning-formal 

SI  - structured  inquiry 

GI  - group  investigation 

RP  - role  playing 

JI  - jurisprudential  inquiry 

NT  - nondirective  teaching 

ES  - enhancing  self-esteem 

ML  - mastery  learning 

PS  - programmed  schedule 

DI  - direct  instruction 

S - simulation 


Teaching  Styles 
(Mosston  & Ashworth,  1990) 

C - command  (A) 

P - practice  (B) 

R - reciprocal  (C) 

SC -self  check  (D) 

I - inclusion  (E) 

GD  - guided  discovery  (F) 

CD  - convergent  discovery  (G) 
DP  - divergent  production  (H) 
LD  - learner  designed  (I) 

LI  - learner  initiated  )J) 

ST -self  teach  (K) 


Teaching  Styles 
(Grasha,  1996) 

E - expert 
FA  - formal  authority 
PM  - personal  model 
F - facilitator 
D - delegator 


Kolb’s  Learning  Styles 
(Kolb,  1984) 

CE  - concrete  experience 
RO  - reflective  observation 
AC  - abstract  conceptualization 
AE  - active  experimentation 


Bloom  (1956) 

K-R  - knowledge  or  remember 

C-U  - comprehension  or  understanding 

Ap  - application  or  apply 

An  - analyze 

S-E  - synthesize  or  evaluate 

E-C  - evaluate  or  create 

Dale’s  Cone  of  Learning 
(Dale,  1969) 

P - passive  (listening  only) 

I - intermediate 

(participating  in  discussion) 

A -active  (doing) 
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Excerpts  from  Questionnaire  - Teaching  Styles  and  Models  Worksheet 

36.  My  professor  exhibited  the  following  styles  of  instruction  throughout  the  semester. 
Select  ALL  that  apply. 

(a) teacher  makes  all  decisions  on  what,  where,  when,  and  how  learning  takes  place; 

is  the  expert;  strives  for  precision,  synchronization,  and  uniformity;  determines 
what  is  taught  and  how  it  will  he  evaluated 

(h) students  are  given  a number  of  tasks  to  practice;  students  can  ask  questions; 

teacher  moves  around  and  gives  feedback 

(c)  students  provide  feedback  to  each  other;  one  student  performs  while  another 

provides  feedback;  teacher  designs  forms  to  guide  the  observations;  socialization 
is  inherent  in  this  style;  students  develop  feedback  skills 

(d)  feedback  is  provided  by  you  as  the  individual  learner  to  yourself;  other  events 

providing  external  feedback  facilitate  your  ability  to  do  this;  professor  helps  you 
become  a better  evaluator,  increasing  your  self-esteem  about  working 
independently 

(e)  we  select  our  own  level  of  performance  and  alter  it  according  to  my  self- 

evaluation;  the  professor  determined  the  tasks  and  defined  the  levels  of  difficulty 

(f)  professor  leads  students  to  discover  concept  by  answering  a series  of  questions; 

professor  determines  concepts  and  best  sequences  for  guidance;  friendly 
environment  with  time  to  think  built  into  the  learning  opportunity;  professor 
traces  a series  of  questions  leading  to  the  answer 

(g)  professor  presents  question;  students  use  logical  and  critical  thinking  to  discover 

solutions;  students  determine  questions  to  ask  rather  than  the  professor; 
professors  respect  for  the  process  and  do  not  interfere 

(h)  professor  encourages  students  to  find  multiple  solutions  to  given  problems; 

professor  selects  the  subject  and  designs  the  problem  - there  is  no  one  right 
answer;  professor  responds  to  student  process  rather  than  the  value  of  a solution 
or  answer 

(i)  the  student  and  professor  selects  the  content  to  be  learned;  the  student  designs, 

develops,  and  performs  the  series  of  tasks  and/or  students  select  the  activity, 
design  the  experiences,  perform  the  tasks;  professors  assists/consults  with  the 
evaluation  of  tasks 

(j)  students  take  full  responsibility  for  the  learning  process;  they  do  not  consult  with 

the  professor 
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Teaching  Models 


The  professor  lectures  information  and  connections;  I listen  and  take  notes,  if  I choose  (Lecture) 

The  professor  focuses  or  presents  content,  then  breaks  the  class  into  student  groups  to  discuss  the 
content,  then  engages  in  summarizing  and  clarifying  the  content  as  a group.  (Reciprocal) 

The  professor  focuses  or  presents  content,  then  assigns  individual  hut  short  term  projects  using 
the  content  or  information,  e.g.,  problem  to  solve,  design  project,  analysis.  (Reciprocal 
Performance) 

The  professor  focuses  or  presents  content,  breaks  the  class  into  student  groups  to  discuss  the 
content,  and  then  engages  in  a short  term  group  project  using  the  content  or  information,  e.g. 
problems  to  solve,  design  project,  analysis.  (Reciprocal  Performance) 

Lessons  are  broken  down  in  components;  as  individual  students  master  each  component,  they  are 
tested;  when  they  pass  the  test,  they  go  on  to  the  next  component.  (Mastery  learning) 

The  professor  uses  visual  charts,  displays  a wide  range  of  graphic  organizers  or  other  visuals  to 
better  organize  and  present  information;  to  show  relationships  between  concepts  and  principles; 
and  to  increase  understanding  about  the  application  of  foundation  concepts  or  principles. 
(Graphic  Organizers) 

When  presenting  content,  the  professor  uses  examples  that  are  and  are  not  representative  of  the 
concept  or  principle.  Students  compare  the  examples  and  match  those  that  represent  the  concept 
or  not;  gradually  as  more  examples  that  are  and  are  not  representative  are  reviewed,  the  group 
reaches  consensus  of  what  examples  directly  represent  the  content  and  come  away  with  greater 
understanding.  (Concept  Attainment) 

Lessons  require  that  we  combine  concepts  and  analyze  the  relationships  of  concepts;  we  then 
engage  in  solving  problems  (conceptualization) 

During  the  lessons,  the  professor  asks  us  to  identify  and  enumerate  information  related  to 
concepts  as  they  are  demonstrated,  grouping  concepts  into  categories  with  common  attributes. 
(Concept  Formation) 

We  learn  information  on  concepts  through  the  act  of  classification,  gathering  and  classifying 
information  to  build  and  test  hypotheses;  they  engage  in  experiments  and  the  results  of 
experiments  are  used  to  develop  hypothesis  generalizations  about  the  situation,  idea,  or  problem, 
(move  from  information  to  problem)  (Inductive  Thinking) 

We  are  presented  with  generalizations  and  examples  and  engage  in  trying  to  glean  or  identify  the 
individual  situation  or  idea  that  is  embedded  (move  from  problem  to  why  something  happens) 
(Deductive  Thinking) 

We  are  presented  with  a problem  and  then  create  questions  to  be  used  to  solve  the  problem.  We 
engage  in  a process  of  investigation  and  explanation  of  the  phenomena.  (Inquiry) 

We  engage  in  a formally  organized  court  case  to  present  information  and  arguments  about  the 
ingrained  issues.  (Jurisprudential) 
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We  are  instructed  on  each  component  of  the  content  and  all  must  he  successful  on  that  content 
before  the  professor  moves  on  with  new  or  more  complex  content  (Direct  Instruction) 

Lessons  break  skills  down  into  components  and  sequences  of  action;  each  person  learns  the  skill 
step  by  step  the  same  way  (Training) 

Lessons  begin  by  focusing  on  a current  situation;  analogies  are  used  to  define  the  characteristics 
of  the  situation;  analogies  continue,  using  other  graduated  analogies  until  it  appears  to  have  no 
relationship  to  the  origin;  the  lesson  then  uses  the  final  description  of  the  analogy  to  compare  to 
the  original  situation  (Synectics) 

Lessons  engage  us  in  the  development  of  physical  skills,  such  as  welding  (Psycomotor) 

The  professor  uses  metaphors  to  make  content  more  familiar  (Metaphorical) 

Lessons  focus  on  personal  development,  free  expression  of  ideas  and  feelings,  furthering  your 
self-understanding  (Non-directive) 

We  explore  problems  through  actions  developing  problem  solving  skills;  we  participate  and/or 
observe  (Role  Play) 
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Bloom's  Taxonomy  of  Learning 


a.  the  learning  of  basic  knowledge  requiring  me  to  list,  name,  identify,  show,  define, 
recognize,  recall,  state,  visualize,  state  facts,  concepts,  theories,  principles,  information? 


b.  the  comprehension  or  greater  understanding  of  knowledge  through  activities  that 
required  me  to  summarize,  explain,  interpret,  describe,  compare,  paraphrase, 
differentiate,  demonstrate,  classify,  or  contrast  facts,  information,  concepts,  theories, 
principles? 


c.  the  application  or  opportunity  to  “do”  or  “perform”  using  knowledge,  requiring  me  to 
solve  problems,  illustrate,  calculate,  use,  interpret,  relate,  manipulate,  apply,  modify 
facts,  concepts,  theories,  information,  or  data? 


d.  analytical  activities  that  required  me  to  analyze  and  organize  facts,  data,  and 
information;  deduce  patterns  and  trends;  and  contrast,  compare,  and  distinguish 
differences  or  similarities,  and  then  discuss  solutions,  directions  and  plan  or  devise 
actions? 


e.  the  synthesis  and  evaluation  of  facts,  information,  data,  situations,  problems,  and 
furthermore  require  me  to  argue  rationally,  support  or  justify  a method,  solution,  action, 
choice  of  formula,  theory,  concept,  principle  or  result  in  the  need  to  propose  a hypothesis, 
following  with  the  design  of  an  experiment,  product,  process,  technique,  and/or  make 
judgments  that  had  to  be  critiqued  and  defended  and  finalized  into  reports,  summaries,  or 
papers. 


f.  the  design,  discovery,  invention,  development,  creation,  research,  or  transformation  of 
knowledge  into  products,  processes,  techniques,  models,  methods,  strategies,  etc.,  using 
design  and  development,  research,  experimentation,  and/or  development  knowledge, 
techniques,  procedures,  and  tools? 
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Degrees  of  Active  or  Engaged  Learning 


a.  The  professor  assumes  the  entire  responsibility  for  delivering  the  course  content. 
He/she  lectures  all  information  we  are  expected  to  learn.  The  text  is  used  as  a reference. 
Lectures  reflect  text  content. 

b.  The  professor  assumes  the  entire  responsibility  for  delivering  the  course  content  in 
combination  with  assigned  readings  from  the  textbook.  The  lectures  and  text  content 
provide  all  the  information  we  are  expected  to  learn.  Most  lectures  correlate  directly  or 
are  duplication  of  text  content. 

c.  Students  are  assigned  reading  from  the  text  to  gain  basic  course  content;  my  professor 
explains  difficult  content  from  the  text,  then  adds  lectures  on  some  important  or  critical 
content  that  is  not  covered  in  the  text,  thus  expanding  or  deepening  understanding  and 
ability  to  use  the  information  from  the  text. 


d.  Students  are  responsible  for  some  of  their  own  learning.  For  example,  once  a concept, 
or  principle  is  explained  by  the  professor  and  we  have  used  the  text  for  basic  learning,  as 
a source  or  reference,  we  then  have  to  perform  research  on  content  ourselves  to  deepen 
our  understanding  of  the  concept  and  its  application  possibilities.  We  have  to  bring  the 
information  back  to  class  to  share  with  the  professor  and  class.  Student  activities  can  vary 
from  literature  research,  case  studies,  identifying  additional  sources  of  information  (e.g. 
books,  people,  examples,  demonstrations,  etc.).  Students  are  required  to  learn  on  their 
own  or  in  small  groups  to  deepen  understanding  or  extend  learning  and  understanding 
beyond  that  presented  by  the  professor  or  established  learning  activities. 


e.  The  professor  assigns  reading  from  the  text,  explains  difficult  content,  and  then 
provides  content  to  deepen  or  extend  the  basic  text  content  or  to  clarify  or  explain  content 
not  well  understood.  Students  are  responsible  for  some  of  their  own  learning,  and  we  then 
engage  in  research  to  solidify  understanding  of  the  content.  Ultimately,  the  professor  then 
assigns  projects  that  expand  learning  into  the  “doing”  dimension  where  we  used  the 
content  learned  to  solve  a problem,  develop  a product,  construct  a theoretical  model,  use 
materials,  processes,  and  knowledge  to  create,  etc. 

f.  Students  are  responsible  for  a great  deal  of  their  own  learning.  After  working  with  us 
in  a variety  of  ways,  many  of  them  highly  engaging  students  to  learn  important 
knowledge  and  skills  where  the  professor  is  more  of  a learning  coach,  direction  setter, 
source  of  validation,  someone  who  models  an  inquiry  driven  process  of  learning,  with  a 
strong  focus  on  "how"  and  "why"  processes,  he/she  provides  the  opportunity  to  engage 
in  the  creation  of  a solution  to  a identified  need  or  problem  applying  the  knowledge  and 
skills  learned  earlier  or  throughout  the  learning  processes  throughout  the  semester. 
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Teaching  Portfolio  Assessment  Chart,  DAI 

rE  - CITE  Facu 

ty  Development  Program 

Portfolio  Product  (Artifact)  Content 
(See  Sections  of  information  following  this  summary) 

Faculty 

Member 

Faculty 

Member 

Self  Assessment  Baseline: 

1 . Student  Questionnaires  (f05  & f06) 

1 .  Professor  completion(s)  of  Student  Questionnaire  (f05  &f06) 

2.  Professor  completions  of  Self  Competency  Questionnaire 

(Feb.06,  May06,  Dec.06) 

3.  Program  Components  Assessments  (8) 

4.  Standard  Departmental  Course  Evaluations  (f05  & f06) 

4. Student  Grades  & End  of  Semester  Grades  (f05  & f06) 

5.  Course  Analysis: 

5al._Course  Outline,  Embedded  Gen  Ed,  Content  Priorities 
5a2._Course  Content  Analysis  by  TM,TS,  LS,  Bl,  Dale,  etc. 

5b.  _ Instr.  Design  GAPS  Analysis  on-  TM,  TS,  LS,  B,  D 
5c.  _ Instructional  GAPS  Summary 
5d.  _ ABET/TAC/NAIT  SLO  by  Bloom’s  Analysis 
5e.  _ Course  Content  Schedule 

5f._  Teaching  Models+Cooperative  Learning+Study  Chart+TM  graphic 
5g._  Course  Calendar  by  TM,  TS,  LS,  B,  D 

***Student  Learning  Styles  Inventory:***  (NOT  REQUIRED) 
Kolb  (Extra  professional  effort  on  part  of  professors) 

Felder  (Extra  professional  effort  on  part  of  professor) 

Multifaceted  Assessment  System: 

5h. Multifacted  Assessment  Plan  Graphic,  showing  course  assessments 

5i. Test  and  Test  Items  by  SLO  Chart 

5j. Assessment  Analysis  by  Bloom  (Chart) 

6.  Traditional  Objective  Tests:: 

Test  Analysis  (Midterm  and  Final  Exam) 
Table  of  Specifications  (not  included) 
Test  Item  Bank  (not  included) 

7.  New  Midterm  Exam 

7.  New  Final  Exam 

8.  Diagnostic  Write  Ups  (MT  & F) 

Performance  Assessment  & Rubrics: 

7.  3 Complex  Performance  Assessments  with  multiple  tasks 

embedded 

7.  3 Rubrics,  one  to  score  each  Performance  Assessment 

( And  to  be  used  with  students  to  establish  standards  up  front) 

8.  Diagnostic  Write  Ups  (PA  1.2.3) 

* Copies  of  Students  Rubrics  (Hardcopies  on  file) 

7.  Electronic  copies  of  tests  and  PAs  & Rubrics 

other  Assessments  of  Individual  Choice:  Yes  for  All 

9.  Student  Centered  Course  Syllabus: 

All  new  components  and  check  off  list 

10.  Professors’  Research: 

Completed  Data  Forms  (including  data  on  MT,  F,  PA1,2,3) 

Research  Results  Reports 

12. Teaching  Portfolio  Assessment  Questionnaire 

13  Teaching  Models  Self  Assessment 

14  Teaching  Styles  Self  Assessment 

15  Student  Learning  Style  Opportunities  Assessment 

16  Outcomes  Achieved  as  Planned  by  Bloom  & Dale  Assessment 

17.  Manuscript  to  be  submitted: 

Draft 

Final  Version  to  be  submitted  to  journal  tMav.  2007) 

Legend:  J = akay  X = still  needed  © = net  due  yet  See  Result 
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Initial  Teaching  Portfolio  Assessment  Process  May  25,  2006  - 
CITL  Professional  Development  Program  


Portfolio  Product  (Artifact)  Content 

Self  Assessment  Baseline: 

Student  Questionnaire 

Professor  completion  of  Student  Questionnaire 
Professor  Self  Competency  Questionnaire 

Description  of  my  Knowledge, 
Skills,  Abilities  on  Feb.  2,  2006 

Description  of  my  Knowledge, 
Skills,  Abilities  on  May  25, 
2006 

Course  Analysis: 

GAPS  Analysis  on-  TM,  TS,  LS,  B,  D 
ABET/TAC/NAIT  SLQ  by  Bloom’s  Analysis 
Course  Calendar  bv  TM.  TS.  LS.  B.  D 
Teaching  Models  + Cooperative  Learning  + 
Mapping  Study  Chart 

Student  Centered  Course  Syllabus: 

All  new  components  and  check  off  list 

Portfolio  Product  (Artifact)  Content 

Multifaceted  Assessment  System: 

Course  Assessment  Plan  Chart  showing  course 
assessments 

Assessment  Analysis  bv  Bloom  (Charti 

Traditional  Objective  Tests: 

Test  Analysis 

Table  of  Specifications 

Test  Item  Bank 

New  Midterm  Lxam 

New  Pinal  Exam 

Test  Items  bv  SLQ  Chart 

Performance  Assessment  & Rubrics: 

3 Complex  Performance  Assessments  with 
multiple  tasks  embedded 

3 Rubrics,  one  to  score  each  Performance 
( Assessment  and  to  be  used  with  students  to  establish 
standards  up  front) 

Other  Assessments  of  Individual  Choice: 

List  and  Describe  Here 

1.  3. 

2.  4. 
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Initial  Individual  Portfolio  Checklist  of  Products  (Artifacts) 


*Note:  For  each  component  below,  there  is  a rubric,  checklist,  worksheet,  etc.  from  which  to  perform. 
Those  are  available  for  review  in  the  Faculty  Development  Program  section. 

Self  Assessment  Baseline: 

Student  Questionnaire 

Professor  completion  of  Student  Questionnaire(s) 

Professor  Self  Competency  Questionnaire(s) 

Program  Component  Assessments 

Standard  Departmental  Course  Evaluations 

Course  Analysis: 

GAPS  Analysis  on-  TM,  TS,  LS,  B,  D -i-  base  worksheet 

ABET/TAC/NAIT  SEO  by  Bloom’s  Analysis 

Course  Calendar  by  TM,  TS,  ES,  B,  D 

Teaching  Models-i-Cooperative  Eearning-i-Mapping  Study  Chart 

Student  Centered  Course  Syllabus: 

All  new  components  and  check  off  list 

Multifaceted  Assessment  System: 

Course  Assessment  Plan  Chart  showing  course  assessments 

Assessment  Analysis  by  Bloom  (Chart) 

Traditional  Objective  Tests: 

Test  Analysis 

Table  of  Specifications 

Test  Item  Bank 

New  Midterm  Exam 

New  Einal  Exam 

Test  Items  by  SEO  Chart 

Performance  Assessment  & Rubrics: 

3 Complex  Performance  Assessments 

with  multiple  tasks  embedded 

3 Rubrics,  one  to  score  each  Performance 

Assessment  and  to  be  used  with 
students  to  establish  standards 
up  front 

Other  Assessments  of  Individual  Choice: 

List  and  Describe  Here: 

I. 2. 

3. 4. 

Teaching  Portfolio  Assessment  Questionnaire 
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Point  value  for  each  item  is  from  1 to  4 - the  same  as  the  response  labels.  Maximum  possible  points  = 100. 


Competency  Self  Assessment 
CEET  Initiative  on  Teaching  and  Learning 

Jule  Dee  Scarborough  (2006) 

Please  respond  to  each  question  about  the  level  of  knowledge,  skill,  and  confidence  you  feel  you  have. 


1.  Design  and  develop  courses  where  student  learning  objectives  and  outcomes  are  clear  and  distinctly  different. 


1 

Lack  sufficient 
knowledge,  skills, 
confidence  to  apply 


2 3 

Possess  some  knowledge  and  Possess  some  knowledge 

skills,  but  lack  sufficient  and  skills;  feel  somewhat 

confidence  to  apply  confident  to  apply 


4 

Possess  high  level 
knowledge  and  skills; 
confident  to  apply 


2.  Designing  and  developing  a logical  and  organized  course  where  course  syllabi  are  structured  such  that  students 
fully  understand  what  is  to  happen  for  the  entire  semester;  where  the  syllabus  is  the  course  map  for  both  myself 
(professor)  and  the  students;  where  the  syllabus  provides  all  information  about  objectives,  course  content,  timeline, 
course  requirements,  student  learning  assessments  (tests,  projects,etc.)  grading  structure  and  criteria,  etc. 

12  3 4 

Lack  sufficient  Possess  some  knowledge  and  Possess  some  knowledge  Possess  high  level 

knowledge,  skills,  skills,  but  lack  sufficient  and  skills ; feel  somewhat  knowledge  and  skills; 

confidence  to  apply  confidence  to  apply  confident  to  apply  confident  to  apply 

3.  Providing  learning  activities  that  align  with  the  syllabus,  course  content  identified  in  the  syllabus;  adhering  to  the 
timeline  in  the  syllabus;  and  leading  student  learning  without  significant  distractions  or  deviations  unrelated  to 
content  where  each  lesson  and  learning  activity  are  directly  related  and  add  value;  where  no  unplanned,  last 
minute,  or  major  assignments  not  identified  on  the  syllabus  are  imposed  upon  students  unexpectedly. 

12  3 4 

Lack  sufficient  Possess  some  knowledge  and  Possess  some  knowledge  Possess  high  level 

knowledge,  skills,  skills,  but  lack  sufficient  and  skills ; feel  somewhat  knowledge  and  skills; 

confidence  to  apply  confidence  to  apply  confident  to  apply  confident  to  apply 

4.  Designing  a total  student  assessment  system  where  there  is  a great  variety  of  types  of  student  learning  assessment, 
tests,  quizzes,  case  studies  in  industry,  literature  studies,  research,  papers  or  other  writing  assignments,  projects, 
presentations,  portfolios,  etc. 

12  3 4 

Lack  sufficient  Possess  some  knowledge  and  Possess  some  knowledge  Possess  high  level 

knowledge,  skills,  skills,  but  lack  sufficient  and  skills;  feel  somewhat  knowledge  and  skills; 

confidence  to  apply  confidence  to  apply  confident  to  apply  confident  to  apply 

5.  Designing  student  assessments  that  directly  align  and  measure  knowledge  and/or  skills  itemized  on  course  syllabi. 

12  3 4 

Lack  sufficient  Possess  some  knowledge  and  Possess  some  knowledge  Possess  high  level 

knowledge,  skills,  skills,  but  lack  sufficient  and  skills ; feel  somewhat  knowledge  and  skills; 

confidence  to  apply  confidence  to  apply  confident  to  apply  confident  to  apply 

6.  Implementing  a student  learning  assessment  strategy  throughout  the  course  where  feedback  on  all  student 
assignments,  or  learning  assessments  is  immediate  (or  reasonably  timed,  e.g.  2 weeks);  in  other  words,  students 
receive  feedback  from  the  professor  on  grades  or  scores  for  tests,  projects,  etc.  that  can  be  considered  immediate  in 
the  university  schedule  context,  e.g.  1-3  classes  later. 

12  3 4 

Lack  sufficient  Possess  some  knowledge  and  Possess  some  knowledge  Possess  high  level 

knowledge,  skills,  skills,  but  lack  sufficient  and  skills ; feel  somewhat  knowledge  and  skills; 

confidence  to  apply  confidence  to  apply  confident  to  apply  confident  to  apply 
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7.  Designing  and  developing  instruction  using  Bloom’s  Taxonomy  of  levels  of  learning  where  the  upper  levels  of 
Bloom’s  are  achieved  many  times  throughout  the  course,  e.g.  levels -knowledge,  comprehension,  application, 
synthesis,  analysis,  and  evaluation. 

12  3 4 

Lack  sufficient  Possess  some  knowledge  and  Possess  some  knowledge  Possess  high  level 

knowledge,  skills,  skills,  but  lack  sufficient  and  skills ; feel  somewhat  knowledge  and  skills; 

confidence  to  apply  confidence  to  apply  confident  to  apply  confident  to  apply 

8.  Designing  and  developing  student  learning  assessments,  e.g.  tests,  performances,  using  Bloom’s  Taxonomy  of 
levels  of  learning  where  the  upper  levels  of  Bloom’s  are  achieved  many  times  throughout  the  course,  e.g.  levels- 
knowledge,  comprehension,  application,  synthesis,  analysis,  and  evaluation. 


1 2 3 

Lack  sufficient  Possess  some  knowledge  and  Possess  some  knowledge 

knowledge,  skills,  skills,  but  lack  sufficient  and  skills;  feel  somewhat 

confidence  to  apply  confidence  to  apply  confident  to  apply 

9.  Developing  a course  using  the  “reversed  design”  process. 

1 2 3 

Lack  sufficient  Possess  some  knowledge  and  Possess  some  knowledge 

knowledge,  skills,  skills,  but  lack  sufficient  and  skills;  feel  somewhat 

confidence  to  apply  confidence  to  apply  confident  to  apply 


4 

Possess  high  level 
knowledge  and  skills; 
confident  to  apply 


4 

Possess  high  level 
knowledge  and  skills; 
confident  to  apply 


10.  Designing  and  developing  traditional  tests  that  directly  measure  what  students  are  responsible  for  learning  from 
professors  and  other  sources  or  activities  identified  in  the  syllabi;  where  the  items  are  an  adequate  sample  of  the 
content  domain  and  the  item  types  are  appropriate  to  the  purposes  of  the  test;  where  the  items  require  thinking  at 
the  upper  levels  of  Bloom’s  Taxonomy;  where  the  scoring  and  grading  procedures  are  transparent  to  the  student 
and  as  objective  as  possible;  where  each  item  can  be  traced  directly  back  to  the  standards,  student  learning 
objectives,  text  and  source  information,  and  learning  experiences. 


1 

Lack  sufficient 
knowledge,  skills, 
confidence  to  apply 


2 

Possess  some  knowledge  and 
skills,  but  lack  sufficient 
confidence  to  apply 


3 

Possess  some  knowledge 
and  skills;  feel  somewhat 
confident  to  apply 


4 

Possess  high  level 
knowledge  and  skills; 
confident  to  apply 


11.  Performing  test  analysis  involving  statistical  analysis  of  students’  responses;  obtaining  item  statistics  for 
judging  and  improving  the  quality  of  individual  items  and  for  judging  and  improving  the  quality  of  instruction. 

12  3 4 

Lack  sufficient  Possess  some  knowledge  and  Possess  some  knowledge  Possess  high  level 

knowledge,  skills,  skills,  but  lack  sufficient  and  skills ; feel  somewhat  knowledge  and  skills; 

confidence  to  apply  confidence  to  apply  confident  to  apply  confident  to  apply 

12.  Developing  student  performance  tasks  to  measure  what  students  can  do  with  knowledge;  where  there  is  a direct 
link  to  traditional  tests;  which  move  student  learning  assessment  from  “knowing  about”  to  “performing  or  doing- 
using knowledge”;  where  the  upper  levels  of  Bloom’s  Taxonomy  or  levels  of  learning  are  demonstrated. 

12  3 4 

Lack  sufficient  Possess  some  knowledge  and  Possess  some  knowledge  Possess  high  level 

knowledge,  skills,  skills,  but  lack  sufficient  and  skills;  feel  somewhat  knowledge  and  skills; 

confidence  to  apply  confidence  to  apply  confident  to  apply  confident  to  apply 


2 


13.  Designing  and  developing  rubrics  for  the  purpose  of  scoring  (grading)  student  performances  that  measure 
"learning  by  doing"  where  students  demonstrate  what  they  know  by  using  knowledge  to  perform  a task,  etc. 

12  3 4 

Lack  sufficient  Possess  some  knowledge  and  Possess  some  knowledge  Possess  high  level 

knowledge,  skills,  skills,  but  lack  sufficient  and  skills ; feel  somewhat  knowledge  and  skills; 

confidence  to  apply  confidence  to  apply  confident  to  apply  confident  to  apply 

14.  Designing  and  developing  a course  where  many  different  teaching  models  are  used  throughout  the  course,  e.g. 
lecture,  direct  instruction,  reciprocal,  reciprocal-performance,  mastery  learning,  graphic  organizers,  concept 
attainment,  conceptualization,  inductive  thinking,  deductive  thinking,  concept  formation,  inquiry,  training, 
synectics,  psychomotor,  metaphorical,  non-directive,  role  play,  cooperative/collaborative,  etc.. 

12  3 4 

Lack  sufficient  Possess  some  knowledge  and  Possess  some  knowledge  Possess  high  level 

knowledge,  skills,  skills,  but  lack  sufficient  and  skills;  feel  somewhat  knowledge  and  skills; 

confidence  to  apply  confidence  to  apply  confident  to  apply  confident  to  apply 

15.  Designing  and  developing  a course  where  many  different  teaching  styles  are  used  throughout  the  semester,  e.g. 
command,  practice,  self-check,  inclusion,  guided  discovery,  convergent  discovery,  divergent  production,  learner- 
designed,  learner-initiated,  self-teaching  styles. 

12  3 4 

Lack  sufficient  Possess  some  knowledge  and  Possess  some  knowledge  Possess  high  level 

knowledge,  skills,  skills,  but  lack  sufficient  and  skills;  feel  somewhat  knowledge  and  skills; 

confidence  to  apply  confidence  to  apply  confident  to  apply  confident  to  apply 

16.  Designing  and  developing  a course  that  accommodates  different  learning  intelligences  and  student  learning 
styles.  Learning  styles  as:  "characteristic  of  the  cognitive,  affective,  and  physiological  behaviors  that  serve  as 
relatively  stable  indicators  of  how  learners  perceive,  interact  with,  and  respond  to  the  learning  environment"  or 
"the  way  each  learner  begins  to  concentrate,  process,  and  retain  new  and  difficult  information"  or  "concrete 
experience,  reflective  observation,  abstract  conceptualization,  active  experimentation". 

12  3 4 

Lack  sufficient  Possess  some  knowledge  and  Possess  some  knowledge  Possess  high  level 

knowledge,  skills,  skills,  but  lack  sufficient  and  skills ; feel  somewhat  knowledge  and  skills; 

confidence  to  apply  confidence  to  apply  confident  to  apply  confident  to  apply 

17.  Designing  and  developing  a course  where  the  burden  of  learning  is  on  the  student  rather  than  the  professor; 
where  the  professor  assumes  instructional  leadership  and  directs  student  learning,  but  is  not  entirely  responsible  for 
“imparting”  all  the  knowledge  to  be  learned  “directly”;  where  students  engage  actively  in  their  own  learning;  where 
lecture  is  not  KING.  BE  HONEST! 

12  3 4 

Lack  sufficient  Possess  some  knowledge  and  Possess  some  knowledge  Possess  high  level 

knowledge,  skills,  skills,  but  lack  sufficient  and  skills ; feel  somewhat  knowledge  and  skills; 

confidence  to  apply  confidence  to  apply  confident  to  apply  confident  to  apply 

18.  Designing  and  developing  student  learning  activities  where  “cooperative”  or  “collaborative”  learning  models 
are  used  formally;  where  “informal”,  “formal”,  or  “base”  structures  are  used  based  upon  Kagan  or  Johnson  and 
Johnson  strategies  throughout  the  course  or  for  major  learning  activities. 

12  3 4 

Lack  sufficient  Possess  some  knowledge  and  Possess  some  knowledge  Possess  high  level 

knowledge,  skills,  skills,  but  lack  sufficient  and  skills ; feel  somewhat  knowledge  and  skills; 

confidence  to  apply  confidence  to  apply  confident  to  apply  confident  to  apply 
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19.  Designing  and  developing  student  learning  activities  where  cooperative  learning  in  small  groups  provides  the 
opportunity  for  students  to  experience  accomplishing  a goal  together;  where  students  would  feel  that  they  are 
gaining  self-esteem,  respect  from  others,  that  they  are  learning  more  because  they  are  learning  with  others,  that  the 
experience  raised  everyone's  learning  and  consequently  their  grades;  where  higher  level  thinking  occurs  because  of 
students  engaging  in  inquiry  together,  asking  questions  of  each  other;  where  social  skills  develop  as  an  outcome;  and 
they  increase  their  capacity  to  cope  with  stress  or  adversity;  especially  where  group  learning  is  designed  with 
performance  criteria  and  where  they  were  trained  or  educated  about  group  behavior  or  dynamics. 

12  3 4 

Lack  sufficient  Possess  some  knowledge  and  Possess  some  knowledge  Possess  high  level 

knowledge,  skills,  skills,  but  lack  sufficient  and  skills ; feel  somewhat  knowledge  and  skills; 

confidence  to  apply  confidence  to  apply  confident  to  apply  confident  to  apply 

20.  Designing  and  developing  learning  activities  where  multiple  intelligences  are  required  for  learning. 

12  3 4 

Lack  sufficient  Possess  some  knowledge  and  Possess  some  knowledge  Possess  high  level 

knowledge,  skills,  skills,  but  lack  sufficient  and  skills;  feel  somewhat  knowledge  and  skills; 

confidence  to  apply  confidence  to  apply  confident  to  apply  confident  to  apply 

21.  Designing  and  developing  problem-based  learning  where:  students  are  assigned  a problem  with  conditions, 
constraints,  possibilities,  that  require  materials  (sometimes),  research,  collaboration;  where  students  have  to  take 
responsibility  for  their  own  learning  by  solving  the  problem,  where  the  problem  crosses  the  boundaries  of 
disciplines,  entwining  theory  and  practice,  where  there  is  a focus  on  the  processes  of  knowledge  acquisition,  rather 
than  the  products  of  such  processes;  where  the  professor  is  a facilitator  rather  than  instructor,  and  students  will 
engage  in  self  and  peer  assessment. 

12  3 4 

Lack  sufficient  Possess  some  knowledge  and  Possess  some  knowledge  Possess  high  level 

knowledge,  skills,  skills,  but  lack  sufficient  and  skills ; feel  somewhat  knowledge  and  skills; 

confidence  to  apply  confidence  to  apply  confident  to  apply  confident  to  apply 

22.  Engaging  in  the  scholarship  of  teaching,  research  in  the  classroom  on  teaching  and  student  learning  using 
appropriate  research  design  and  methodology,  analytical  or  statistical  procedures,  etc. 


1 

Lack  sufficient 
knowledge,  skills, 
confidence  to  apply 


2 

Possess  some  knowledge  and 
skills,  but  lack  sufficient 
confidence  to  apply 


3 

Possess  some  knowledge 
and  skills;  feel  somewhat 
confident  to  apply 


4 

Possess  high  level 
knowledge  and  skills; 
confident  to  apply 


23.  Using  course  evaluation  data  or  information  as  feedback  to  determine  course  changes. 


1 

Lack  sufficient 
knowledge,  skills, 
confidence  to  apply 


2 

Possess  some  knowledge  and 
skills,  but  lack  sufficient 
confidence  to  apply 


3 

Possess  some  knowledge 
and  skills;  feel  somewhat 
confident  to  apply 


4 

Possess  high  level 
knowledge  and  skills; 
confident  to  apply 


24.  Evaluating  the  effectiveness  of  "interventions"  or  course  changes  to  improve  student  learning. 

12  3 4 

Lack  sufficient  Possess  some  knowledge  and  Possess  some  knowledge  Possess  high  level 

knowledge,  skills,  skills,  but  lack  sufficient  and  skills;  feel  somewhat  knowledge  and  skills; 

confidence  to  apply  confidence  to  apply  confident  to  apply  confident  to  apply 


25.  Closing  the  feedback  loop  and  actually  making  course  changes  for  the  purpose  of  improving  student  learning. 


1 


2 


3 


4 


Lack  sufficient 
knowledge,  skills, 
confidence  to  apply 


Possess  some  knowledge  and  Possess  some  knowledge 
skills,  but  lack  sufficient  and  skills;  feel  somewhat 

confidence  to  apply  confident  to  apply 


Possess  high  level 
knowledge  and  skills; 
confident  to  apply 


4 


Faculty  Participants 

Abul  Azad  ^ ^ / 1 • I 

Brian  Coller  U juove  leachiriQ  at 

Abhijit  Gupta 
Reinaldo  Moraga 
Ibrahim  Abdel-Motaleb 
Regina  Rahn 
Robert  Tatara 


Volume  IV-Section  D:  Presentations 

The  Scholarship  of  Teaching 
The  CEET  Initiative  on  Teaching  and  Learning 


Jule  Dee  Scarborough,  Initiative  Director 

' College  of  Engineering  and  Engineering  Technoiogy 


Promod  Vohra,  Dean 


Program  Associate:  Jerry  Gilmer,  Director,  NIL!  Testing  Services 
Editor:  Gail  Jacky,  Director,  NIL!  Writin'^»4^enter 


■'S- 
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(for  PowerPoint  presentations,  contact  juiescarborough@niu.edu) 


Reflective  Practice 
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The  Scholarship  of  Teaching 


College  of  Engineering  & Engineering  Technology 
Initiative  on  Teaching  and  Learning 
Professional  Development  Program 


Feb.  2 - May  25,  2006 


• Through  learning  we  c ourselves. 


• Through  learning  we  become  able  to  do  something  we 

• Through  learning  we  , to  be 

part  of  the  generative*  process  of  life^^^^^L 


J ull^Searboroui 


(Senge,  1990) 


Faculty  Learning  Communities 


“groups  of  people  gathered  together  intentionally 
for  the  purpose  of  supporting  each  other  and 
the  process  of  learning...” 


Learning  Circles 

“smaller  groups  of  learners  within  the  larger  group  or 
community...  through  professional  interaction  and 
[dialog]...  where  members  construct  ideas 
together.,  share  opinions.,  debate  issues” 


J u|^  Scar  boroug  h 


I 


Collay  etal.  (1998) 


Global  Communities  of  Practice  and  Interest 


Regional  & National  Public  Communities  of  Interest 


Business  & Industry  Communities  of  Practice 


Jule  Scarborough,  2006 


Faculty  Learning  Communities 


a shared  goal,  problem  or  project; 

shared  resources; 

shared  membership  and  leadership; 

commitment  to  improvement  of  professional 
practice; 

coliaborative  approaches  to  groupwork; 

iearning  and  deveiopment  focused  on  real  work- 
based  Issues  and  practice; 

autonomous  community  members' 

high  levels  of  dialogue,  interaction  and 
coliaboration; 


information  and  knowiedge  sharing; 

knowiedge  constructivism; 

knowiedge  transfer  and  knowledge  exchange; 

use  of  information  and  communication  technologies. 
(p.6-7) 


Communities  of  Practice 

• “Companies  at  the  forefront  of  the  knowledge 
economy  are  succeeding  on  the  basis  of 

“professionar  communities  of  practice... 

• Communities  of  practice  are  groups  of  people 
who  share  a concern,  a set  of  problems,  or  a 
passion  about  a topic,  who  deepen  their 


\NE  ARE  A COMMUNiTY  OF  PRACTiCE! 


Wenger,  McDermott,  and  Snyder,  (2002) 


Learning  Environments 


J u|^S€ar  ttoroug  h 


rSyjifi 


. Knowledge 
i i centered 


Learner 

centered 


Assessment 

centered 


Bransford  et  ai  (1998) 


Learning  is  NOT! 


Static 

• Listening,  memorizing,  and  then  repeating  what  we  recalled 

• Where  we  provide  learners  with  the  answers  rather  then  the 
questions  and  problems... 

Learning  |S! 

ILOynamic 

R Society,  the  real  world,  requires  learners  to  actively  plan, 
)bserve,  test,  and  reflect,  rather  than  be  passive.... 


5\ctive,  inquiry-based,  learning  puts  the  burden  of  learning  on 
the  learner  to  explore  the  unknown 


Jull^Scarhofough 


Thomas  etal,  (2005) 


Learning  IS  NOT  synonymous  with 
“taking  in  information” 

*Generative  Learning 

“learning  that  enhances  our  capacity 

to  create” 


Ju'I^Scarboroug  hj 


Transformative  Learning 


Grants  power  to  the  learner. . . 
to  relate  to  the  subject  matter 
to  build  upon  existing  knowledge 
to  construct  new  knowledge,  and... 
Empowers  one  to  create  their  desired  future 


(Senge,  2000) 


Learning  = Shift  of  Mind= 


-<From  seeing  ourseives 
-■as  Separate  from  the  worid 

• to.. -Connected  to  the  worid 

• to... Transforming  the  worid 

• to... Creating  the  worid 


Pedagogy 


Transmissron  Pedagoav-g^ 

from  the  learner  and  the  teadW 


takes  power  away 


Pedagogy  - grants  the  power  to 
to  the  subject  matter  and^Jjfll»ion  existing 
Rnowledge  for  both  the  teacher  and  the  learner 


Transformative  Pedagogy  - grants  the 
HESS  one’s  desired  future  for  both  teachers  and 
learners 


(Senge,  2000) 


Self-assessment 

Does  our  teaching  have  purpose,  meaning,  focus? 


Critical  Questions: 

- Do  our  courses  have  integrity? 

- Do  our  assessment  strategies/procedures  have  integrity? 

- Do  our  teaching  processes  have  integrity? 

- Do  our  educational  products  have  integrity? 

- Do  we  as  professors  have  integrity? 

- Do  our  syiiabi  have  integrity? 

- Are  we  creating  a iearning  communities  of  students  where 
they  can  “make  meaning”  of  new  knowledge? 

- Do  we  deliver  on  “promises”  to  students? 

- Are  we  “transforming  ieaders”  ?(Bums,  1978)  _ _ 


Jule  Scarborough, 
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Syllabus  Analysis 


Teaching  Analysis 
2/20-2/24 


Bloom’s  Analysis 
Dale’s  Analysis 


Assessment  Analysis 


Knowledge  Analysis 


Faculty  Instructional 
Analysis 


Develop 

Performance  Tasks 
& Rubrics 

+ Other 
Assessments 


Grading 


>■ 

- 


Research 

& 

Evaluation 


Design 

Methodology 

5/25 


Develop 
Instructional 
Strategies 

T 

I 


Conduct 
Experiments 
Eall  2006 
Collect  Data 


The  Sct]olarship  of  Teaphing 

Reflective  Practice  \ 

I CEETs  I 

Faculty  'pevelopment  Program 
‘Intentional  ^nd  Reversed”  Instructional 

Design  ' 


Analyze, 

Interpret 

Data 

Report 

Results 


T 


‘Backward”  Desigi 
McTighe,  2005,  ^ 


L ^lannini 


(Scarborough,  2006)  \ 

Process  (Dick  & Carey,  1996:  Wiggins  & 
1995;  Deming  Quality  Cycl^e,  l950”s) 
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Revise  Knowledge 

T 

I 

Revise 

Revise 

Content 

I 

Assessments 

Instijuction 

Revise  Research 
QuestioJis/ 
Research  Design 


Ongoing  Change  Loop 


/'TScting'Srx 

^ApalyziruK 
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Eaculty 
Development 
Program 


Identify 
Changes 
Needed  to 
Instruction 


I 


N^Qdjyziijg/ 


’^achieved 


Jule  Scarborough,  2006 


Program 


Learning  Communities 
Scholarship  of  Teaching 

Course  Analysis 

Student  Learning  Outcomes  ABET/TAC/NAIT  Outcomes  Test  Items 
Analyzed  by  Dale  and  Bloom 

Traditional  Test  Analysis  and  Development 
Performance  Assessment  and  Rubric  Development 
Other  Assessments 
Map  of  Assessments 

Teaching  Models  (20  Joyce,  Weil,  and  Calhoun) 

Cooperative  Learning  (Johnson,  Johnson,  and  Smith) 

Concept  Mapping  as  a model  or  assessment  process  (multiple  sources,  styles, 
etc.) 


Multiculturalism  in  Courses 

Grading  - What  do  they  mean  and  how  should  it  be  done  with  integrity 
Experimental  Research  on  Teaching  and  Learning 


Jil(gScar^orough_ 


CEET  Initiative  on  Teaching  & 

Learning 

The  CEET  Scholarship  of  Teaching  Initiative  (CITI)  is  based  upon  the  following: 

CEET  Vision:  To  build  a regional  and  national  reputation  for  the  scholarship  of  teaching. 

CEET  Mission:  To  build  an  interdisciplinary  team  of  faculty  who  understand  the  four  types  of 
scholarship  as  defined  by  Boyer  (1990),  have  the  capability  of  engaging  in  either  or  several  of  the 
types  of  scholarship,  are  who  are  stimulated  to  engage  in  scholarship  activities,  research  on  teaching 
and  student  learning  in  their  disciplinary  classrooms. 

CEET  Promotion  and  Tenure  Objective:  To  redefine  scholarship  across  its  department  and  make 
acceptable  for  promotion  and  tenure  purposes,  either  or  a mix  of  productivity  across  the  types  of 
scholarship  as  defined  by  Boyer  (1990). 

CEET  Goal  for  Faculty:  To  engage  in  scholarship  of  teaching,  either  as,  or  alongside  their  other 
scholarship  interest(s). 


CEET  Scholarship  Goal:  To  adopt  standards  for  quality  performance  in  scholarship.  (Glassick  et  al, 
1997). 

CEET  Goal:  To  institutionalize  and  sustain  a program  of  faculty  development  on  teaching,  student 
learning,  and  educational  research  to  prepare  faculty  to  engage  in  scholarship  of  teaching  through 
action  research  in  the  classrooms. 


CEET  Goal  for  Students:  To  develop  student  learning  communities  where  learners  in  the  truest 
sense  to:  engage  actively  in  constructing  broader  knowledge  frameworks,  make  deeper  meanings, 
increase  transfer  across  contexts  and  that  these  students  leave  us  with  such  excitement  about  what 
they  have  learned  that  they  continue  to  seek  to  learn,  extending  what  they  learned  with  us  into  new 
meanings  through  their  new  experiences  and  opportunities  throughout  their  career. 


Ji  ul^SMrborough^ 


Faculty  Learning  Community 


"transformative"  learning  communities  are 
those  which  enable: 

"like-minded  people,  colleagues,  or 
professionals  with  a common  or  multi- 
professional interest  to  work  together  and  to 
achieve  a particular  aim  or  organizational 
objective." 


Expected  Outcomes: 

1 . Research  action  and  results  to  base  further  research  upon 

2.  Faculty  enabled  to  engage  in  the  scholarship  of  teaching 

3.  The  redefinition  of  the  relationship  of  teaching  to  research  with  the 
adoption  of  modified  bylaws  that  will  reflect  the  four  areas  of 
scholarship  as  those  proposed  by  Boyer  (1990):  discovery, 
integration,  application,  and  teaching. 


Products: 

1.  CEET  faculty  professional  development  model  and  program 

2.  Faculty  leadership  team 

3.  Classroom,  or  student  learning,  pilot  research  results 

4.  New  or  revised  T/L  educational  products,  e.g.  student 
assessments,  syllabi,  models  and  processes,  feedback 
instruments,  others  TBD. 

5.  National  publications  and  presentations 
(to  be  submitted  with  intent  of  acceptance) 

6.  Proposals  for  on-going  research 

(to  be  submitted  with  intent  of  acceptance) 

7.  Faculty  leadership  team 

8.  Proposals  to  modify  faculty  evaluation  procedures  to  include 

scholarship  of  teaching  as  an  acceptable  form  of  research  towards 
promotion  and  tenure  ■i.uj.a.iii.i.i 


Teaching  Professionals 

1 . emphasize  learning  rather  than  teaching. 

2.  emphasize  active  student  engagement  with  significant  content. 

3.  focus  on  student  performance  and  production. 

4.  routinely  collaborate  with  their  colleagues. 

5.  are  students  (themselves)  of  teaching  and  consumers  of  research. 

6.  function  as  leaders  for ... 

"every  great  leader  is  clearly  teaching  and  every  great  teacher  is  leading" 

(Gardner,  1986,  p.  19) 


(Schlechty,  1997) 


Jul^ecirborough^ 


1 0 % of  what  we  READ 


20%  of  what  we  HEAR 


30  % of  what  we  SEE 


Nature  of  Involvement 


Hearing  Words 


Looking  at  Pictures 


Verbal  Receiving 


50  % of  what  we 
HEAR  & SEE 


Watching  a Movie 
Looking  at  an  Exhibit 
Watching  a Demonstration 
Seeing  it  Done  on  Location 


Visual  Receiving 


70  % of  what  we 
SAY 


Participating  in  a Discussion 
Giving  a Talk 


Receiving/ 

Participating 


90  % of  what 
we  SAY  & 
DO 


Doing  in  a Dramatic  Presentation 
Simulating  the  Real  Experience 
Doing  the  Real  Thing 


Dale’s  Cone  of  Learning  (1969) 


Doing 


Bloom  Revised  Taxonomy  of  Learning 

Anderson  & Krathwohl,  (2001) 

The 

Knowledge 

Dimension 

The  Cognitive  Process  Dimension 

1.  Remember 

2.  Understand 

3.  Apply 

4.  Analyze 

5.  Evaluate 

6.  Create 

A. 

Factual 

Knowledge 

B. 

Conceptual 

Knowledge 

C. 

Procedural 

Knowledge 

D. 

Meta-Cognitive 

Knowledge 

Scholarship 

Scholarship  of  discovery 


Knowledge  for  knowledge  sake  - the  creation  of  a bank  of 
knowledge  or  information,  ready  to  draw  upon  when  the 
time  for  intelligence  use  arrives. 


Schoiarship  of  Integration 


Authenticating  knowledge  through  analysis  and 

interpretation,  establishing  meaning  or  original  research 
through  interdisciplinary  consideration  and  synthesis. 


Ju^^Searborouih 


Scholarship 


Scholarship  of  Application 

Where  scholarship  connects  theory  and  practice 
and  proves  its  worth  to  the  nation  and  world. 

Scholarship  of  Teaching  ... 

Where  scholarship  connects  or  develops  theory 
and  practices  on  teaching  and  learning  for  the 
increase  of  student  learning  and  the 
improvement  of  teaching 


J u^^Scarborough^ 


"Teaching  is  the  Highest  Form  of 

Understanding." 

Good  teaching  means  that  faculty,  as  scholars,  are 
also  learners  ...  researchers  about  learning 

transforming  and  extending  knowledge  through 
scholarship 

Without  the  teaching  function,  the  continuity  of 
knowledge  will  be  broken  and  the  store  of  human 
knowledge  dangerously  diminished. 


Boyer  (1990) 


The  Scholarship  of  Teaching  and 
Learning  in  Engineering  and  Technology 

• Scholarship  of  Discovery  - historically  primary 
“A  scientist  discovers  that  which  never  was...” 

An  engineer  creates  that  which  never  was....” 

Historically,  design  was  a linear,  morphological 
process... formal  process 

Recently,  design  is  less  formal... designs  are 
generated  through  a more  social  process 


Principles  to  Improve 
Undergraduate  Education 

Good  Practice: 

> Principle  1 : encourages  student-faculty  contact 

> Principle  2:  encourages  cooperation  among 

students  - Cooperative  Learning 

Communities 

Principle  3:  encourages  Active  Learning 
Principle  4:  gives  students  prompt  feedback 
Principle  5:  emphasizes  time  on  task 

Principle  6:  communicates  high  expectations 

Principle  7:  respects  diverse  talents  and 

ways  of  learning 


Changing  Contexts 


NSF  began  to  shift  funding  away  from  single-investigator 
research  to  multidiscipiinary  centers  — 

resulting  in  iegitimizing  schoiarships  of  integration  and 
appiication  to  address  important  nationai  problems,  e.g. 

energy  production,  environmental  science,  technoioy, 
biotechnology... 

Students  are  more  diverse 


The  Journal  of  Engineering  Education  has  become  the 
primary  source  on  engineering  education  in  the  worid... 


it  focuses  on  educational  principles  as  well  as  educational 
classroom  experiments  (1991) 


Scarborough 


lOfM 


Research-Teaching  and  Learning 

• Advanced  because  of  NSF  programs 

• New  Engineering  Criteria  2000 

(emphasize  standards  and  assessment  of  learning  outcomes) 

• Led  to  substantial  increase  in  scholarship  of  teaching  and 
earnjng  with  ultimate  goal  of  improving  teaching  and 
earning... 

• Engineering  schools  are  beginning  to  focus  on  how  to 
prepare  faculty  to  implement  new  teaching  and  assessment 
methods 


Faculty  development  programs  critical  to  the  development 
of  faculty  and  grad  learning  communities  focused  on  STL 


NSF  program  Engineering  Education  Scholars,  etc. 


gBcarboroughj 


The  Scholarship  of  Teaching  and 
Learning  in  Engineering  and  Technology 

Most  data  from  Engineering  Scholarship  of 
teaching  and  learning  is  qualitative 

Need  more  experimental/control  studies 
Will  improve  research  credibility 
NSF  wants  experimental  research 

Social  science  research  is  “messy”  - quite 
different  from  typical  engineering  research 
Requires  engineering  adjustment 


U ut^Searborouih^ 


Assessing  Educational  Scholarship 


Felder  (2000)  suggests  that  faculty  reviews  for 
promotion  should  ask: 

1 . To  what  extent  does  the  instructor’s  teaching 
quaiify  as  a schoiarly  activity? 

2.  How  effective  is  the  instructor’s  teaching? 

3.  How  numerous  and  effective  are  the  instructors 
_ educational  research  and  development  efforts? 


Piaget 

The  principal  goal  of  education  is  to: 

“create  people  who  are  capable  of  doing  new 
things,  not  simply  repeating  what  other 
generations  have  done... 


People  who  are  creators,  inventors,  discoverers. 


The  second  goal  of  education  is  to: 


“form  minds  that  are  critical,  can  verify,  and  do 
not  accept  everything  they  are  offered 


Reflective  Practice 


PROBLEM 

IDENTtPfCATiON 


OBSERVATION 


ACTIVE 

experimentation 


PROBLEMATIC 

EXPERIENCE 


ANALYSIS 


ABSTRACT 

reconceptualization 


Figure  1.1  Ri‘IIcc(i\'L’  IVacticc:  An  Exporientivil  Learning  Cycle 
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Sullivan  & Glanz,  2006 


Time 


Performance 


FigUre'6.1:^  Anjountrof  Time  SperitsPrepafing^iRelative  to  Te^chingfQpality^ 


Shally  in  Frost  & Taylor  (1996)  p.  69 
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Fig.  14.2  The  Evaluation  Cycle 


Hounsell  in  Fry  et  al  (2003)  p.  210 
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Fig.  1.2  Contrasting  Approaches  to  Professional  Development 


Fig.  2.3  Finding  Discrepancies:  Developing  a Critical  Perspective  on  Practice 


i Describe  Espousals 

Espoused  Theory  Intended  Action  Coals 

What  are  my  explicit  beliefs?  How  did  I intend  to  act?  What  do  I hope 

to  accomplish? 


[Examine  Practice 

Action  Outcomes 

What  did  I do?  What  occurred  as  a 

result  of  my  actions? 

~Contrast  Espousals  with  Practice 

Theory-in-Use  Action  Outcomes 

What  ideas  shape  Did  I act  as  I intended?  Were  my  actions 

my  actions?  effective? 


Examine  Personal  Practice  in  Context  of  Theory  and  Research 

Theory  Action  Predicted  Outcomes 

What  strategies  predictably  lead  to  desired  outcomes? 

What  does  research  tell  us  about  good  practice? 

Is  my  practice  "best  practice”? 


J u|^S<iarbaroui  h 


mm. 


Osterman  & Kottkamp  (2005)  p.  36 


Fig.  4.1  Contrasting  Model  I and 
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Organizational 
Consequences 
Cniiimitment 
Problem-solving 
Creativity  & Learning 


Key  Strategies 

Communicate  Openly 
Emphasize  Common  Coals 
Publicly  Test  Assumptions 
Advocacy  & Inquiry 

Key  Assumptions 
Collaboration 
Problem  Solving 


Dsterman  & Kottkamp  (2005)  p.  70 


Scholarship  of  Teaching 

Learning  Community  - 
Professionai  Deveiopment  Program 
Execute  Program 

Coiiect  data  on  iearning: 

- Student  assessment 

- Self  assessment 

- Competency  assessment 

- Program  content  assessments 

-Teaching  Portfolios  - exhibits  of  change  & 
growth 


J u^^Scarboroug 


Scholarship  of  Teaching 


Where  are  We? 

Questions  about  Experimental  Research!!! 

(with  control  groups  in  intact  classes) 

- Design 

- Methodology 

- Student  Fairness 

- Process 

- Results  --  what  If  we  don’t  show  significant 
scores? 


J ul^Scarborough^ 


Scholarship  of  Teaching 


Pilot  or  Research? 


Tool  Box  on  Teaching  & Learning 

• Includes: 

8 Fjeld  Prioritized  Books 


100+  Articies  - what  we  couidn’t  buy  with 
Dean  Vohra’s  budget,  we  copied! 

We  quit  counting! 

Assessment  Tools 


Program  Material’s 


J ul^Scarboroug  h 
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Teaching  Portfolio 


Self  Assessment  Baseline: 

Student  Questionnaire 

Professor  compietion  of  Student 

Questionnaire 

Professor  Seif  Competency 

Questionnaire 


Course  Anaiysis: 

GAPS  Anaiysis  on-  TM,  TS,  LS,  B,  D 

ABET/TAC/NAiT  SLO  by  Bioom’s 

Anaiysis 

Course  Caiendar  by  TM,  TS,  LS,  B,  D 

Teaching  Modeis+Cooperative 

Learning+Mapping  Study  Chart 


Student  Centered  Course  Syiiabus: 

Aii  new  components  and  check  off  iist 


Muitifaceted  Assessment  System: 

Chart 

Assessment  Anaiysis  by  Bioom 

(Chart) 

Traditionai  Objective  Tests: 

Test  Anaiysis 

Tabie  of  Specifications 

Test  Item  Bank 

New  Midterm  Exam 

New  Finai  Exam 

Test  items  by  SLO  chart 

Performance  Assessment  & Rubrics: 

3 Compiex  Performance 

Assessments 

with  muitipie  tasks  embedded 

3 Rubrics,  one  to  score  each 

Performance 

Assessment  and  to  be  used  with 
students  to  estabiish  standards 
up  front 

Other  Assessments  of  Individuai  Choice: 


Jul^Searborough 


aom. 


Final  Teaching  Portfolio  Report 


Teaching  Portfoio  as 
Assessment 

• Reflection  and  Response  to  Cues 

• Showcase  & Growth 


J u^^Scarboroug  h 
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Scholarship  of  Teaching 


Classroom  Research 


Tentative  Research  Designs 

study  1 

Traditional  Assessment  With  Performance  Assessment 


1.  Does  a performance  test  administered  in  conjunction  with  a 
traditional  cognitive  test  result  in  increased  learning  beyond 
the  traditional  test  alone  as  indicated  by  the  traditional  cognitive 
test? 


2.  Does  a performance  test  administered  in  conjunction  with  a 
traditional  cognitive  test  result  in  increased  knowledge 
retention  beyond  the  traditional  test  alone  as  indicated  by  a 
final  exam? 


Jule  Scarborough,  2006 


Tentative  Research  Designs 

Study  1 : 

Traditional  Assessment  With  Performance 

Assessment 


Study  1:  Traditional  Assessment  With  Performance  Assessment 


Treatment 

Posttest  I 

Posttest  2 

Experimental 

Group 

Instruction 

Performance  Test 
Related  to 
Traditional  Test 

Traditional  Test 

^ 

Traditional  Final 

Control  Group 

Instruction 

Performance  Test  Not 
Related  to 
Traditional  Test 
(Placebo) 

Traditional  Test 

^ 

Traditional  Final 

Jule  Scarborough,  2006 


Tentative  Research  Designs 

Study  2 

Performance  Assessment  and  Traditional 
Assessment  Administered  in  Different  Order 

1 . Does  a performance  test  administered  in  conjunction  with  a 
traditional  cognitive  test  result  in  increased  learning  beyond 
the  traditional  test  alone  as  indicated  by  the  traditional  cognitive 
test? 


2.  Does  the  order  of  administration  of  a performance  test  and  a 
traditional  test  affect  knowledge  retention  as  indicated  by  a 
final  exam? 


Jule  Scarborough,  2006 


Tentative  Research  Designs 


Study  2: 

Performance  Assessment  and  Traditional 
Assessment  Administered  in  Different  Order 


Group  1 

Instruction 

Performance 

Test 

Traditionai 

Test 

^ 

Finai  Exam 

Group  2 

Instruction 

Traditionai 

Test 

Performance 

Test 

^ 

Finai  Exam 

Jule  Scarborough,  2006 


Tentative  Research  Designs 

Study  3 

Passive  Learning  vs.  Active  Learning 


1 . Does  passive  learning  vs.  active  learning  result  in 
differential  knowledge  gains  as  indicated  by  a traditional 
cognitive  test? 

2.  Does  passive  learning  vs.  active  learning  result  In 

differential  knowledge  retention  as  indicated  by  a final 
exam? 


Jule  Scarborough,  2006 


Tentative  Research  Designs 

Study  3: 

Passive  Learning  vs.  Active  Learning 


Study  3:  Passive  Learning  vs.  Active  Learning 

Treatment 

Posttest  1 

Posttest  2 

Passive 

Learning 

Group 

Passive 

Learning 

Traditional  Test 

^ 

Final 

Active 

Learning 

Group 

Active  Learning 

Traditional  Test 

^ 

Final 

Treatment 

Group 

Content 
Area  I 

Content 
Area  II 

Content 
Area  III 

Content 
Area  IV 

1 

Passive 

Active 

Passive 

Active 

2 

Active 

Passive 

Active 

Passive 

Jule  Scarborough,  2006 


Tentative  Research  Designs 

Study  4 

Individual  Learning  vs.  Cooperative  Learning 

1 . Does  individual  learning  vs.  cooperative  learning  result  in 
differential  knowledge  gains  as  indicated  by  a traditional 
cognitive  test? 

2.  Does  individual  learning  vs.  cooperative  learning  result  In 

differential  knowledge  retention  as  indicated  by  a final 
exam? 


Jule  Scarborough,  2006 


Tentative  Research  Designs 

study  4 

Individual  Learning  vs.  Cooperative  Learning 


Treatment 

Posttest  1 

Posttest  2 

Individual  Learning 
Group 

Individual  Learning 

Traditional  Test 

^ 

Final 

Cooperative 
Learning  Group 

Cooperative  Learning 

Traditional  Test 

^ 

Final 

Treatment 

Group 

Content  Area  I 

Content  Area  II 

Content  Area  III 

Content  Area  IV 

1 

Individual 

Cooperative 

Individual 

Cooperative 

2 

Cooperative 

Individual 

Cooperative 

Individual 

Jule  Scarborough,  2006 


Next  Steps  for  Learning  Community 

for 

Schoiarship  of  Teaching  to  Begin  Formaiiy 


• Consider  “piiots”  vs.  “research” 

• Determine  FUTURE  of  Formai  Learning  Community  - Formaiize  operations 

• Read  and  discuss  research  thoroughiy  - engage 

• Study  “TOOLBOX”  Contents!! 

• Consider  support  needs 

• Begin  to  write  articies  based  upon  iearning  so  far 

• Begin  to  “consider”  proposais 

• if  Research: 

“Tweak”  courses  for  experiments  - formaiize  treatments 


Jule  Scarborough,  2006 
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Jule  Scarborough,  2006 


Universities 


Teaching  vs.  Research 


The  Dilemma 
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Regional  & National  Public  Communities  of  Interest 


Business  & Industry  Communities  of  Practice 


■Tiile  Scarhorniigh,  ^2006) 


Jule  Scarborough,  2006 


Professors’  Professional  Responsibility  & Duty 

- Teaching  “Duty”  = “core  of  universities  mission 

& the  faculty’s  academic  duty 

- Scholarship  “Discovery”,  “Integration”, 

“Application”,  “Teaching” 

- Service 

As  teaching  professors,  it  is  our  job  to: 

- remove  barriers  to  learning 

- create  learning  environments  where  students  can 
succeed 

- create  new  avenues/paths  for  learning 

- be  professional 

• use  best  practices  list 

• base  teaching  on  research 

• engage  in  “The  Scholarship  of  Teaching” 

• engage  in  “action”  research 

• others 


As  professors... 


What  is  our  product? 


Who  is  our  customer? 


Public  Demands  ! 


National 

Illinois  Citizens  Agenda-  Goal  2 

- “Higher  education  will  join  elementary  and  secondary  education 
to  improve  teaching  and  learning  at  all  levels  ” 

An  Illinois  Directive! 


Improved  Teaching  and  Student  Learning  K-  University 


Research  on  Teaching  & Learning 


Scholarship  of  Teaching 

- Formal  research 

Action  Research 

- Informal  and  evaluative  inquiry 


Systematic  Instructional  Design 


A 


Identify 

desired 

results 


^^Reversed^^  Instructional 
Design  Model 


Determine 

acceptable 

evidence 


Plan  learning 
experiences 
and  instruction 


Jule  Scarborough,  2006 


Wiggins  & McTighe  (1995) 


Program  Quality  Feedback  Loop 


rf 


Carnegie 
Institute 
On  Teaching 


Literature 

Study 


Che  ;king-Re-planniijg 


Develop 
Faculty 
Development 
Program 


Orientation 
2/2/06 


Faculty 

Self 

Assessment 


Student 

Assessment 

of 

Instruction 


Course 

Evaluations 


Course 

Grades 


Test 

Grades 


Assess 

Needs 


student  Learning 
Style  Analysis 


Multiculturalism 


Learner  Motivation 

f—T 


Write  Learning 
Objectives 


ill 


Syllabus  Analysis 


Teaching  Analysis 
2/20-2/24 


Bloom’s  Analysis 
Dale’s  Analysis 


Assessment  Analysis 


Knowledge  Analysis 


Faculty  Instructional 
Analysis 


iDevelop  Student  Assessments 


Develop 

Tests 


Develop 

Performance  Tasks 
& Rubrics 

+ Other 
Assessments 


Grading 


Research 

& 

Evaluation 


Design 

Methodology 

5/25 


Develop  ’’ 
Instructional  ‘ 
Strategies 

-f 

I 


- 


Conduct 
Experiments 
Eall  2006 
Collect  Data 


The  Scfiolarship  of  Tei^ching 

Reflective  Practice*^ 

I CEETs  I 
Faculty  Development  Prl^gram 
“Inteiptional  and  Revetted” 
Instructional  Design 

I (Scarborough,  2006)  \ 

“Backwardr  Design  Process  (Dick  & Carey,  1996; 
Wiggins  and  MtfTighe,  200^  1995;  Deming  (Quality  Cycle 


Analyze, 

Interpret 

Data 

Report 

Results 


T 


> 

Revise  Knowledge  | 

Revise 

Revise 

Content  | 

Assessments 

Instijuction 

Revise  Research 
Questiolis/ 
Research  Design 

1 


L ^lannini 


Ongoing  Change  Loop 


Identify 
Changes 
Needed  to 
Eaculty 
Development 
Program 


Identify 
Changes 
Needed  to 
Instruction 


I 


’^achieved 


Jule  Scarborough,  2006 


Learning  Organization-  CELT 

❖Faculty  Learning  Community 

Faculty  Learning  Circles/Teams 

Faculty  & Student  Learning 

Communities 

Student  Learning  Communities 
Student  Learning  Circles/  Teams 
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Regional  & National  Public  Communities  of  Interest 


Business  & Industry  Communities  of  Practice 


Jule  Scarborough,  2006 


Learning  Organizations 


Universities . . . 

“consciously  structured  to  promote  their  own 
learning  and  that  of  their  students  and 
faculty  members. . rather  than 


individual,  isolated,  passive  learning” 


Tinto  (1997)  in  Lenning  and  Ebbers,  (1999). 


Learning  Community 


“ An  intentional  means  to  a particular  end, 
which  is  to  maximize  learning  of  groups  and 
individuals  ” 


Lenning  and  Ebbers,  (1999). 


-Faculty  Learning  Communities 


-Student  Learning  Communities 

-Faculty  & Student  Learning 
Communities 

-Learning  Circles 

• ^^consciously  and  proactively  structured 
groups  organized  primarily  to  promote 
learning’’ 


Critical  Conditions  for  LCs  & Ics 


1 . Building  community 

2.  Constructing  knowledge 

3.  Supporting  learners 

4.  Documenting  reflection 

5.  Assessing  expectations 

6.  Changing  cultures 


Collay  et  ah  (1998) 


Communities  of  Learners . . . 


• “groups  of  people  gathered  together  intentionally 
for  the  purpose  of  supporting  each  other  and  the 
process  of  learning. . . 

• Learning  Circles 

“smaller  groups  of  learners  within  the  larger  group  or 
community. . . where  there  is  a more  personal  forum  for 
professional  interaction  and  greater  opportunity  for 
conversations  [dialog] . . . where  members  can  construct 
ideas  together.,  share  opinions.,  debate  issues” 


Collay  et  ah  (1998) 


Members 


Take  responsibility 

Make  choices  and  decisions 

Agree  on  a agenda  with  leaders 


Professional  Cultures 

for 

Faculty  & Students 

1 . Shared  decision-making 

2.  Shared  sense  of  purpose 

3.  Collaborative  work  towards  that  purpose 

4.  Collective  responsibility 


Learning  Communities 

Require 

Transformative  Communication 


Both  senders  and  receivers  of  information 
are  changed  as  they  engage  in  the  learning 
experience 


Characteristics  of  Learning 
Communities: 

• A shared  goal,  problem  or  project 

• Shared  resources 

• Shared  membership  and  leadership 

• Commitment  to  improvement  of  professional  practice 

• Collaborative  approaches  to  group  work 

• Learning  and  development  focused  on  real-world  based 
issues  and  practice 

• Autonomous  community  members 

• High  levels  of  dialogue,  interaction  and  collaboration 

• Information  and  knowledge  sharing 

• Knowledge  constructivism 

• Knowledge  transfer  and  knowledge  exchange 

• Use  of  information  and  communication  technologies 

Lewis  and  Allan,  (2005) 


Learning  Communities. . . 

“ the  challenge .. .to  take  advantage  ofLC 
structure  to  capture  and  intensify  the  synergistic 
possibilities  for  meaningful  community  building 
and  learning. . . 

space  to  bring  together  theory  and  practice 

Core  Practices 

community  - diversity 

integration  - active  learning 

make  meaning  - reflective  assessment 


Critical  Questions 


• Why  do  we  exist? 

• What  is  our  fundamental  purpose? 

• What  do  we  hope  to  become? 

• What  do  we  hope  to  achieve? 

• What  are  our  strategies  for  becoming  better? 

• How  will  we  define  our  expected  results  reality? 

• What  criteria  will  we  use  to  assess  our 
improvement  effort? 


Communities  of  Practice 


• ''Companies  at  the  forefront  of  the 
knowledge  economy  are  succeeding  on  the 
basis  of  communities  of  practice,  whatever 
they  call  them...  communities  of  practice  are 
groups  of  people  who  share  a concern,  a set 
of  problems,  or  a passion  about  a topic, 
who  deepen  their  knowledge  and  expertise 
in  this  area  by  interacting  on  an  ongoing 
basis.  ” 


Jule  Scarborough,  2006 


Characteristics  of  Communities  of  Practice 

( real  world  business,  industry,,. organizations) 


• common  purpose  identified  by  participants 

• shared  membership  and  leadership 

• participants  likely  to  be  at  different  stages  in  their 
professional  life 

• acceptance  of  low  levels  of  participation  by  new 
members,  that  is,  legitimate  peripheral 
participation 

• development,  creation  and  management  of 
knowledge  within  organizations 

• open-ended,  not  time  bound 

• importance  of  dialogue,  interaction  and  shared 
narratives 

Jule  Scarborough,  2006 


Teaching  - truly,  one  of  the 
grandest  of  adventures 


Are  we  the  “learned”  or  the 
“learning”? 

(One  experiment  after  another) 


Jule  Scarborough,  2006 


Professionals  Who  Teach 


1 . emphasize  learning  rather  than  teaching 

2.  emphasize  active  student  engagement  with 
significant  content 

3.  focus  on  student  performance  and  production 

4.  collaborate  with  colleagues 

5.  are  students  (themselves)  of  teaching  and 
consumers  of  research 

6.  function  as  leaders  for  “ every  great  leader  is 
clearly  teaching  and  every  great  teacher  is 
leading  ” 


Jule  Scarborough,  2006 


Learning  - What  does  it  mean 


Jule  Scarborough,  2006 


• How  do  I know  my  students  have  learned? 

• How  do  I know  I’ve  learned? 

• What  provides  evidence  of  their  learning? 

• What  provides  evidence  of  my  learning? 


Jule  Scarborough,  2006 


• “meta”-  above  or  beyond  + “noia”(nous)  - of  mind 

= transcendence  of  mind 

• “a  fundamental  shift  or  movement  of  mind” 

• the  deeper  meaning  of  learning 


Jule  Scarborough,  2006 


(Senge,  1990) 


Through  learning  we  re-create  ourselves. 


• Through  learning  we  become  able  to  do  something 
we  never  were  able  to  do  [before] . 

• Through  learning  we  extend  our  capacity  to  create, 
to  be  part  of  the  generative*  process  of  life. 


Jule  Scarborough,  2006 


(Senge,  1990) 


Learning  IS  NOT  synonymous  with 
“taking  in  information” 


Jule  Scarborough,  2006 


Knowledge 


• . a process,  and  a relational  one  at  that,  which 

cannot  therefore  be  located  simply  in  a individual 
head,  to  be  extracted  and  shared  as  an 
organizational  asset.  Knowledge  is  the  act  of 
conversing.  And  learning  occurs  when  ways  of 
talking,  and  therefore  patterns  of  relationship, 
change. . .the  knowledge  assets  of  an  organization 
lie  in  the  pattern  of  relationships  between  its 
members” 


Jule  Scarborough,  2006 


Stacey  (2001) 


Learning  i 


• Listening,  memorizing,  and  then  repeating  what 
we  recalled 

• Where  we  provide  learners  with  the  answers  rather 
I then  the  questions  and  problems . . . 


Dynamic 


Society,  real  world,  requires  learners  to  actively 
plan,  observe,  test,  and  reflect,  rather  than  the 
passive  approach  above. 

Active,  inquiry-based,  learning  puts  the  burden  of 
learning  on  the  learner  to  explore  the  unknown 


Static 


Jule  Scarborough,  2006 


Thomas  etaly  (2005) 


Shift  of  mind  leads  one: 


• from... 

- Seeing  ourselves  as  separate  from  the  world 

• to... 

- Connected  to  the  world 


• to... 

- Transforming  the  world 

• to... 

- Creating  the  world 

Jule  Scarborough,  2006 


(Senge,  1990,  2000) 


Assumptions 


Learning  : 

• Inquires  into  underlying  assumptions  and 
deepens  the  learning  process 

• Is  an  active  process  that  occurs  over  time 

• Is  driven  by  the  learner  around  meaningful 
issues 

• Is  experimental  by  nature 


Jule  Scarborough,  2006 


Wald  and  Castleberry,  (2000) 


Professionals  Construct  Self  Knowledge 


learning  must  acknowledge  and  build  on  prior  experiences  and 
knowledge.  Accordingly,  professionals  need  opportunities  to  explore, 
articulate,  and  represent  their  own  ideas  and  knowledge 


learning  is  more  effective  when  it  takes  place  as  a collaborative  rather 
than  an  isolated  activity  and  in  a context  relevant  to  the  learner. 


Jule  Scarborough,  2006 


Knowledge  Community  Beliefs 


(For  all  information’s  independence  and  extent,) 
it  is  people,  in  their  communities,  organizations,  and 
institutions  ultimately 


Knowledge  is  something  we  digest  rather  than 
merely  hold.  It  entails: 

- the  knower’s  understanding  and  having  some  degree  of 
commitment 


Jule  Scarborough,  2006 


* Generative  Learning 


“learning  that  enhances  our  capacity  to 

create” 


Jule  Scarborough,  2006 


(Senge,  1990) 


“holding  a vision  and  concurrently  telling  the 
truth  about  the  current  reality  relative  to  that 
vision” 


Results  in 

- Individuals  influencing  their  reality 


Jule  Scarborough,  2006 


(Senge,  1990;  Martin  Luther  King  Jr.,  1986;  Socrates,  ?) 


• Deep  down  we  are  all  learners 


•We  hunger  for. . . and  love  to. . . learn 


Jule  Scarborough,  2006 


(Senge,  1990) 


Transformative  Learning 


- grants  the  power  to  relate  to  the  subject  mater 


- builds  upon  existing  knowledge 


- constructs  new  knowledge 


- empowers  one  to  create  their  desired  future 


Jule  Scarborough,  2006 


(Senge,  2000) 


Pedagogy 


- Transmission  Pedagogy-  takes  power  away 
from  the  learner  and  the  teacher 

- Generative  Pedagogy  - grants  the  power  to 
relate  to  the  subject  matter  and  build  on  existing 
knowledge  for  both  the  teacher  and  the  learner 


- Transformative  Pedagogy  - grants  the  power  to 
create  one’s  desired  future  for  both  teachers  and 
learners 


Jule  Scarborough,  2006 


(Senge,  2000) 


Learning... 


-begins  with  a question 


-requires  openness  about 

and 

-comfort  with  “not  knowing 


Jule  Scarborough,  2006 


After  2 Weeks  We 
Remember  This 


Nature  of  Involvement 


1 0 % of  what  we  READ 
20%  of  what  we  HEAR 


30  % of  what  we  SEE 


Reading 


Verbal  Receiving 


Hearing  Words' 


Looking  at  Pictures 


50  % of  what  we 
HEAR  & SEE 


^ Watching  a Movie 
r Looking  at  an  Exhibit 


Visual  Receiving 


hing  a Demonstration 
Seeing  it  Done  on  Location 


70  % of  what  we 
SAY 


Participating  in  a Discussion 
Giving  a Talk 


Receiving/ 

Participating 


90  % of  what 
we  SAY  & ' 


DO 


Doing  in  a Dramatic  Presentation 
Simulating  the  Real  Experience 
Doing  the  Real  Thing 


Doing 


Dale’s  Cone  of  Learning  (2001) 
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K 


Reading 


Hearing  Words 


K 


Looking  At  Pictures 
Watching  A Movie 


c 


Looking  At  An  Exhibit 
Watching  A Demonstration 
Seeing  It  Done  On  Location 


c 


Participating  In  A Discussion 
Giving  A Talk 


C 


Doing  Dramatic  Presentation 
Simulating  The  Real  Experience 
Doing  The  Real  Thing 


ASj 
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Bloom’s  Revised  Taxonomy  of  Learning 

Anderson  & Krathwohl,  (2001) 


The 

Knowledge 

Dimension 

The  Cognitive  Process  Dimension 

1 . Remember 

2.  Understand 

3.  Apply 

4.  Analyze 

5.  Evaluate 

6.  Create 

A. 

Factual 

Knowledge 

B. 

Conceptual 

Knowledge 

C. 

Procedural 

Knowledge 

D. 

Meta-Cognitive 

Knowledge 

Jule  Scarborough,  2006 


Self-assessment 


• Does  our  teaching  have  purpose,  meaning,  focus? 


Critical  Questions: 

- Do  our  contents  have  integrity? 

- Do  our  measurement  strategies  and  procedures  have 
integrity? 

- Do  our  teaching  processes  have  integrity? 

- Do  our  educational  products  have  integrity? 

- Do  we  as  professors  have  integrity? 

- Do  our  syllabi  have  integrity? 

- Are  we  training  or  educating? 

- Are  we  creating  a learning  community? 

- Do  we  deliver  on  “promises?” 

- Are  we  “transforming  leaders”  (Bums,  1978) 


Do  all  students  have  to  do  the  same  thing  to 

learn  the  same  thing? 


Don’t  forget  to  ask  your  students. 


Student  Learning 
Communities 
(Teams) 
Technology  496 


ule  Scarborougl 
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Jule  Scarborough,  2006 
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5? 
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Assessment 


Jule  Scarborough,  2006 


D JZove  teaching  at 


NIU 


N O K r H K K N 
ILLINOIS 
INIVFRSIT^ 


Community  of 
Practice  Partners 


Operations  Engineers 


1 


<1D  .JZove^  ^~Ce^cLc}vina  cti 


N O K I III  K X 
II  1 I N (>  I > 

I N I V I R % I r > 


Student  Learning  Outcomes 


(for  PowerPoint  presentations,  contact  juiescarborough@niu.edu) 


Student  Learning  Outcomes 


The  Terminology  and  History 


Jule  Scarborough,  2006 


Intentional  Instructional  Design 

Reversed  Design  Process 


What  do  you  want  students  to  know? 

What  do  you  want  students  to  be  able  to  do? 

What  is  acceptabie  evidence  of  learning? 

How  do  you  want  students  to  provide  evidence 
of  learning? 


Jule  Scarborough,  2006 


Dick  and  Carey,  1996 
Wiggins  and  McTighe,  2005 


Systematic  Instructional 


lanning) 


[^Research 

Carnegie 
Institute 
On  Teaching 


Program  Quality  Feedback  Loop 


earner  Analysis 


student  Learning 
Style  Analysis 


Develop 
Faculty 
Development 
Program 


Faculty 

Self 

Assessment 


iDevelop  Student  Assessments 


Develop 

Tests 


student 

Assessment 

of 

Instruction 


Course 

Evaluations 


Course 

Grades 


Test 

Grades 


Assess 

Needs 


Teaching  Analysis 
2/20-2/24 


Bloom’s  Analysis 
Dale’s  Analysis 


Assessment  Analysis 


Knowledge  Analysis 


Faculty  Instructional 
Analysis 


Develop 

Performance  Tasks 
& Rubrics 

+ Other 
Assessments 


Grading 


’’  Research 
& 

Evaluation 


Design 

Methodology 

5/25 


Develop  ^ 
Instructional 
Strategies 

I 


Conduct 
Experiments 
Fall  2006 
Collect  Data 


The  Scholarship  of  Teaching 

'Reflective  Practice 

I CEETs  I 

Facum  Development  Program 
“Intentional  and  Reversed”  Instructional 
1 Design  1 

' (Scarborough,  2006)  [ 

“Backwarf’ Design  Process  (Dick S^Carey,  1996; 

Wiggins  & McTighb,  2005,  1995,  Denting  Quality  Cycle  (1950’s) 


Analyze, 

Interpret 

Data 


Report 

Results 


T 


Revise  Knowledge  ^ 

Revise 

Content  ^ 

Assessments 

Revise 

Insti^uction 

1 


I 


_ fflannin: 


Ongoing  Change  Loop 


Identify 
Changes 
Needed  to 
Faculty 
Development 
Program 


Identify 
Changes 


Needed  to 
Instruction 


I 


N^cHyzirifK 


’^achieved 
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student  Learning  Objectives 

or  Outcomes 


Objectives  should  be  STUDENT-CENTERED 


T 

students  will  be  able  to  demonstrate  what 

they  know  or  can  do 


Jule  Scarborough,  2006 


Hernon  et  al  2004,  p.48 


Terminology  and  Meanings 


• Behavioral  Objectives? 

• Course  Objectives? 

• Student  Learning  Objectives? 

• Learning  Goals? 

• NOW  WHAT!? Are  these  terms  interchangeable? 

• Student  Learning  Outcomes??? 

(See:  Joanna  Allan  “Learning  Outcomes  in  Higher  Education”. 
Studies  in  Higher  Education,  Vol.  21,  No.1, 1996,  pp.93-107) 


Jule  Scarborough,  2006 


Student  Learning  Objectives  or 

Outcomes 


embedded  within  the  broader  standards  and 
goals  of  the  accreditation  or  educational 
agencies,  university,  and  programs 

Results  Oriented 

outcomes  of  learning  can  be  observed  and 
measured 


Jule  Scarborough,  2006 


Learning  Expectations 

• Specificity  increases  at  each  level 

• Upper  level  goals,  e.g.  institutional/agency 

establish  areas  of  expected  knowledge  or  skill 
(do  not  specify  discipline,  conditions,  learning 
level) 

• Student  learning  objective  or  outcome,  most 
specific  level  (should  specify  discipline, 
conditions,  learning  levels) 


Jule  Scarborough,  2006 


Specificity  of  Objectives 
A Continuum 

• Global  - broad,  complex,  multifaceted  learning 
outcomes  that  require  time  and  instruction  to 
accomplish,  e.g.  “excite  the  imagination” 

• Educational  - more  focused,  delimited 
“the  ability  to  read  musical  scores” 

• Instructional  - focused  on  monthly,  weekly,  day 
to  day  “slices”  of  learning,  fairly  specific  in 
content,  e.g.  “able  to  differentiate  among  four 
international  business  theories” 

Anderson  & Krathwohl  (2001) 


Jule  Scarborough,  2006 


Accreditation  Standards  & State  Board  of  Higher  Education  Goais 
(Giobai) 


NIU  Generai  Education  Goais 
(Giobai) 


Coiiege  and  Departmentai  Program  Goais 
(Educationai  -disc  piine  or  sub-discipiine) 


Course  Objectives 

(Instructionai  -conditions,  constraints,  ieveis  of  iearning) 


Student  Learning  Objectives 
& Outcomes 


Juie  Scarborough,  2006 


Examples 


Institutional  (global)  learning  outcomes: 

Students  will  be  capable  of: 

analysis,  problem  solving,  communicating, 
decision-making;  will  have  a global  perspective, 
be  capable  of  social  interaction,  etc. 

Students  will  be  capable  of:  reading,  writing, 
reasoning,  acquiring  knowledge,  etc. 


Jule  Scarborough,  2006 


What  Objectives  are  NOT! 

Means  vs.  Ends 

Ends:  objectives  de^cribes^^ep#^^  intended 
results,  outcomesT^^  changes^S*^  students 

Means  to  the  End:  instructional  activities,  e.g. 
reading  the  text  book,  listening  to  the 
professor,  conducting  an  experiment,  going 
on  a fieldtrip 

Anderson  & Krathwohl  (2001) 


Jule  Scarborough,  2006 


Relationship  of  Global,  Educational,  and 
Instructional  Objectives 


Level  of  Objective 

Global  Educational  Instructional 

Scope 

Broad 

Moderate 

Narrow 

Time 

Needed 

One  or  more  years 
(often  many) 

Weeks  or  months 

Hours  or  days  or 
weeks 

Purpose  or 
Function 

Provide  vision 

Design  curricuium 

Prepare  iesson  pians 

Exampie 
of  Use 

Anderson  & Krathwohi 
(2001) 

Pian  a muitiyear 
curricuium,  B.S. 
dearee  in 

Technoiogy  or  B.S. 
in  Engineering 

Pian  units  of 
instruction.  Course 
in  Generai  Ed.,  B.S. 
in  Engineering  or 
Technoiogy 

Pian  monthiy,  weekiy, 
daiiy  activities, 
experiences,  and 
exercises  in  a course 

Jule  Scarborough,  2006 

Planning  Student  Learning 


Desired 

Outcome 

Methods  of 
Assessment 

Level  of 
Assessment 

Strategy 

Recommendations 

Changes  in 
Learning 

What  knowledge, 
abilities,  or  skills 
should  students 
exhibit? 

Competencies? 

What  level  of 
competency  do  1 
expect  of  them? 

What  evidence 
of  learning  will 
be  required? 

Is  it  ^ 

direct^ 

or  indirect? 

Which  method? 
Why? 

Is  assessment  at 
the  institutional 
program,  or 
course  level? 

What  levels  of 
learning  will  be 
required? 

Are  there 
options? 

How  will 
students  be 
given  the 
opportunity 
to  gain  that 
knowledge  or 
skill? 

Based  on  the  evidence 
gathered,  what  are  the 
suggestions  for 
improving  student 
learning? 

What  resulted 
from 

implementation 
of  the 

recommendatio 

ns? 

Anderson  & 

Krathwohl  (2001) 

Jule  Scarborough,  200 

Test 

Performances 

others 

5 

Example: 
Layered 
Curriculum^ 
Choices  A-C 

Lecture 

Text 

other  Methods 

Based  upon  how  well  my 
student  learned,  what  am  I 
going  to  change  to 
increase  student  learning 
the  next  time? 

Did  students 
learn  at  higher 
levels? 

Were  they  more 
competent? 

J 

Identify 

desired 

results 

^^Backward^^  Instructional 
Design  Model 


Determine 

acceptable 

evidence 


Plan  learning 
experiences 
and  instruction 


Jule  Scarborough,  2006 


Wiggins  & McTighe  (1995) 


Program  Research  Outcome  & Embedded  Competencies 

To  apply  relevant  research  to  problem-centered  course  assignments. 

1 . Analyze  a problem  in  its  full  complexity. 

2.  Identify  and  relate  relevant  literature  to  the  problem  under 
investigation. 

3.  Select  or  develop  a theoretical  framework  appropriate  to  solving  that 
problem. 

4.  Select  an  appropriate  procedure  (research  design  and  methodologies) 
that  addresses  study  objectives,  research  questions,  and  hypotheses. 

5.  Adopt  appropriate  indicators  of  reliability  and  validity. 

6.  Demonstrate  effective  written,  oral,  and  presentation  skills  to  convey 
study  findings  and  how  those  findings  match  the  stated  problem 

statement.  Hernon  04,p.304 


Jule  Scarborough,  2006 
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Assessment  Analysis 
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Instructional 
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Fall  2006 
Collect  Data 


The  Scholarship  of  Teaching 
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I CEETs  I 

Faculty  Development  Program 
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1 Design  1 

[ (Scarborough,  2006)  [ 

“Backward^” Design  Process  (Dick  &'^Carey,  1996; 

Wiggins  & McTighd,  2005,  1995,  Denting  Quality  Cycle  (1950’s) 


Analyze, 

Interpret 

Data 


Report 

Results 


T 


> 

Revise  Knowledge  ^ 

Revise 

Content  ^ 

Assessments 

Revise 

Insti^uction 

1 


I 


_ fflannin: 


Ongoing  Change  Loop 


Identify 
Changes 
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Faculty 
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Program 
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Learning  Objective 

“pre-fornnulated  specific  goal”  - educational  intent  or  goal  of 
learning  - what  the  professor  “intends”  to  teach  and  student 
should  learn  (Eisner,  1 979, p. 103) 

measurable,  achievable,  tangible,  observable  product  of 
learning  which  is  capable  of  being  specified  in  advance 

terms  that  identify  both  the  behaviour  to  be  developed  and  the 
content  within  which  this  behaviour  is  to  operate 

“to  write  clear  and  well-organized  fuel  cell  research  reports” 

NOT 

topics,  content,  and  concepts;  these  fail  to  indicate  what 
students  are  expected  to  do  with  the  content;  does  not 
specify  what  is  supposed  to  ensure  from  learning  experience 
(e.g.  course  outlines  generally  part  of  a course  syllabus). 


Jule  Scarborough,  2006 


Behavioral  Objective 


kinds  of  changes  in  behaviour  (thinking,  feeiing, 
action)  that  an  institution  seeks  to  bring  about  in  its 
students  (inciuding  conditions  and  standards-Mager, 
1962) 


“to  be  abie  to  to  write  a musicai  composition  with  a 
singie  tonai  base  within  four  hours.  The  composition 
must  be  at  ieast  16  hours  iong  and  must  contain  at 
ieast  twenty-four  notes.  You  must  appiy  at  ieast  3 
ruies  of  good  composition  in  the  deveiopment  of 
your  score”  [performance  task] 

(Eisner,  1979) 


Jule  Scarborough,  2006 


Behavioral  Objective 

• To  state  an  objective  that  will  successfully 
communicate  your  educational  intent,  you  will 
sometimes  have  to  define  terminal  behavior  further 
by  stating  the  conditions  you  will  impose  upon  the 
learner  when  he/she  is  demonstrating  mastery  of  the 
objective,  what  he/she  is  to  do  or  perform  while 
demonstrating  mastery: 

“Given  a standard  set  of  tools....” 

“Given  a matrix  of  intercorrelations...” 

“Given  a linear  algebraic  equation  with  one 
unknown,  the  learner  must  be  able  to  solve  for  the 
unknown  without  the  aid  of  references,  tables,  or 

calculating  devices.”  Jule  Scarborough,  2006 


Learning  Outcome 

“essentially  what  one  ends  up  with,  intended  or  not,  after  some  form  of 
engagement”  [learning  event] 

what  the  student  achieves;  consequences  of  learning  experiences 

(Eisner,  1979,  p.103) 

should  be  subject-specific  content  and  context 

“that  on  completion  of  the  research  methods  module,  the  student  will  be 
able  to  apply  knowledge  of  validity,  reliability  and  trianquiation  to  a 
chosen  research  issue.” 


(shows  what  the  student  will  be  able  to  do  as  a result  of  learning 
experiences  that  have  been  planned) 

“what  learners  know  and/or  can  do  as  a result  of  learning”  (Otter  1992,  p.i) 
[know  about  vs.know] 


Jule  Scarborough,  2006 


Verbs 


• Words  open  to  many 
interpretations: 

To  know 

To  understand 

To  really  understand 

To  appreciate 

To  fully  appreciate 

To  grasp  the  significance  of 

To  enjoy 

To  believe 

To  have  faith  in 


• Words  open  to  fewer 
interpretations: 

• To  write 

• To  recite 

• To  identify 

• To  differentiate 

• To  solve 

• To  construct 

• To  list 

• To  compare 

• To  contrast 


Jule  Scarborough,  2006 


Standards 

• Statements  about  what  students  are  expected  to 
learn... 

• “essential  skills” 

• “learning  expectations” 

• “learning  outcomes” 

• “achievement  expectations” 

• “other  names 

(Nitko,  2004) 


Jule  Scarborough,  2006 


Content  vs.  Knowledge 

Content: 

subject  matter  content  or  content  domain 
matter  dealt  with  in  a fieid 
substance 

Who  determines  the  “content  substance”? 

“those  scholars  who  have  spent  their  lives  studying 
and  working  in  a fieid,  e.g.  mathematics,  engineers, 
etc.  - shared  knowledge”  Over  time,  they  share, 
change,  extend  knowledge  through  interaction 
throughout  the  schoiarly  community;  fields  are  not 
static,  rather  dynamic  as  changes  are  made  when 

new  idea  and  evidence  are  generated...  Anderson  & Krathwohl  (2001) 

Jule  Scarborough,  2006 


Fig.  7.11  Curricular  Priorities  & Assessment  Methods 

In  effecX-'we  a&sessmen'ts.  we  see  a match  between  the  type  or  format  of  the  aa&e&&menX^  and  the  needed 
evidence  of  achieving  the  desired  results.  If  the  goal  is  for  students  to  learn  basic  facts  and  ©kills,  then  paper- 
and-pencil  tests  and  c^uizze©  generally  provide  adeg^uate  and  efficient  measures.  However,  when  the  goal  is 
deep  understanding,  we  rely  on  more  complex  performances  to  determine  whether  our  goal  has  been  reached. 
The  graphic  below  reveal©  the  general  relationship  between  assessment  types  and  the  evidence  they  provide 
for  different  curriculum  targets. 


Wiggins  & McTighe  (2005)  p.  170 


Fig.  3.3  Clarifying  Content  Priorities 


Wiggins  & McTighe  (2005)  p.  71 


Knowledge  vs.  Subject  Matter  Content 


Knowledge  - disciplinary  subject  matter,  e.g. 

concepts,  principles,  facts,  theories  in  an 
academic  discipline 

Subject  Matter  Content  - materials  used  to 
convey  the  knowledge  and  promote  learning 
to  the  students,  “the  packaging”,  e.g. 
textbooks,  courses,  multi-media  packages  - 
curricular  or  instructional  materials  - materials 

Anderson  & Krathwohl  (2001) 

Jule  Scarborough,  2006 


Standards 


• Content  Standards 

statements  about  subject  matter  facts, 
concepts,  principles,  etc. 

• Performance  Standards 

statements  about  the  things  students  can 
perform  or  do  once  the  content  is  learned 

(Nitko,  2004) 

Jule  Scarborough,  2006 


KSAs 


• Knowledge  (K) 

• Skills  (S) 

• Abilities  (A) 

• What  we  are  trying  to  develop  in 
students... 

• NOT  JUST  Knowledge  and  Skills 

Jule  Scarborough,  2006 


Learning  Targets 


• Standards 

• Goals 

• Objectives 


Learning  Targets 
Learning  Targets 
Learning  Targets 


Specific  Learning 
Targets 


Mastery  Learning 
Targets 


(Nitko,  2004) 


Jule  Scarborough,  2006 


Mastery  Learning  Targets 

Statements  of  what  student  can  do  at  the  end  of 
instruction 

“Can  do”  statements  “To  Verb  Noun  + ” 

(conditions,  constraints,  etc.) 


Also  called:  “specific  learning  outcomes” 

“behavioral  objectives” 

(refer  to  definition  above-oid  expression) 

(Nitko,  2004) 


Jule  Scarborough,  2006 


Developmental  Learning  Targets 


• Skills  and  abilities  more  aptly  stated  at  a 
somewhat  higher  level  of  abstraction 
than  mastery  learning  targets 

• Why? 

• To  communicate  that  they  are 
continuously  developed  throughout  life. 


(Nitko,  2004) 


Jule  Scarborough,  2006 


Examples  of  Developmental 
Learning  Targets 


1 . Analyze  and  make  critical  judgments 
about  the  viewpoints  expressed  in 
passages. 

2.  Write  several  paragraphs  that  explain 
the  author’s  point  of  view. 


Jule  Scarborough,  2006 


Developmental  Objectives 


4.  Use  numerical  concepts  and 
measurements  to  describe  real-world 
objects. 

5.  Interpret  statistical  data  found  in 
material  from  a variety  of  disciplines. 

6.  Write  imaginative  and  creative  stories. 


Jule  Scarborough,  2006 


Developmental  Objectives 


7.  Use  examples  from  materials  read  to 
support  your  point  of  view. 

8.  Communicate  your  ideas  using  visual 
media  such  as  drawings  and  figures. 


(Nitko,  2004) 


Jule  Scarborough,  2006 


Developmental  Learning  Targets 

• They  are  generic 

• No  conditions  “lifelong  in  nature” 

• No  constraints 

• No  specific  discipline 

• No  specified  levels  of  learning 

(Nitko,  2004) 


Jule  Scarborough,  2006 


Lifelong  & Course  Specific 


Need  both 

Can  embed  one  in  the  other 


Jule  Scarborough,  2006 


Taxonomies  of  Learning 

• Bloom’s  Taxonomy  of  Learning 

Domains:  Cognitive  Affective  Psvchomotor 

Cognitive  Domain  - knowledge,  skills,  and  abilities 
requiring  memory,  thinking,  and  reasoning 

Affective  Domain  - feeiings,  interests,  attitudes, 
dispositions,  and  emotional  states 

Psvchomotor  Domain  - motor  skiiis  and 
perceptuai  processes 


(Nitko,  2004) 


Jule  Scarborough,  2006 


Bloom ’s 
Taxonomy  q 
Co 

Levels^’ 
Bloom  (1956) 
Nitko  (2004,  p.24-30) 


Evaluation 

-students  make 
judgments 
relative  worth  of 
’ ideas,  products  & positions 
as  they  take  positions  & justify 

^ Synthesis 

-students  exploit  their  mastery  of 
concepts  to  create  new  and  original 
products,  ideas  and/or  solutions 
Analysis 

-students  take  apart  concepts,  ideas  and/or  products 
in  order  to  identify  patterns  & relationships 

Application 

students  are  able  to  exploit  their  understanding  of  factual  material 
to  solve  problems  and  design  and  make  products 

Comprehension 

-students  make  sense  of  factual  material  and  are  able  to  employ  this 
understanding  to  represent  the  information  in  a variety  of  media  and  or  genres 

Knowledge 

-students  engage  factual  material  and  able  to  recall  and  describe  facts 
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Others  - Nitko,  Appendix  D & E 

pp.  460-467 

• Bloom  et  al 

• Gagne’ 

• Quellmalz 

• Anderson  & Krathwohl,  et  al  (Revised  Bloom) 

• Harrow 

• Categories  of  Learning  Targets  Derived  from 

the  Dimensions  of  Learning  Modei  (pp.467- 
469) 


(Nitko,  2004) 
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Learning  Taxonomies 


Don’t  teach  to  them 

They  are  NOT  teaching  hierarchies 

Purpose:  to  ciassifv  iearning  targets 

to  ciassifv  assessment  tasks 


(Nitko,  2004) 


Jule  Scarborough,  2006 


Learning  Taxonomies 

For  example,  you  should  NOT: 

...teach  “knowledge”  first, 

...teach  “comprehension”  second 
...teach  “application”  third,  etc. 


(Nitko,  2004) 


Jule  Scarborough,  2006 


Objectives  and  Outcomes 

What  is  the  intended  meaning  of  an  objective? 

Do  they  represent  ? .... 

OR  Performance? 

*Assessed  student  performance  is  used  to  make  inferences 
about  intended  student  learning.** 

*The  more  specific  an  objective,  the  easier  it  is  to  assess! 


Jule  Scarborough,  2006 


Some  Best  Practices 

Benchmarking 

Compare  your  courses  against  others,  national  colleagues,  for  content,  teaching  models  and  styles, 
student  assessments,  learning  experiences,  etc. 


Content  Validity  Checks 

Send  your  course  syllabi  out  to  individuals  in  related  communities  of  practice,  e.g.  business, 
industry,  hospitals,  employers  of  your  students 


Authenticity  Checks 

Check  course  knowledge  content,  student  assessments  against  those  in  the  real  world 
communities  of  practice 

Continuous  Improvements 

Close  the  loop  with  each  course,  each  semester;  consider  course  outcomes  achieved,  those 
not  achieved  as  well  as  desired;  identify  needed  or  desired  changes;  incorporate  changes  - 
that  closes  the  feedback  and  change  loop.  Try  again. 


Jule  Scarborough,  2006 
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(for  PowerPoint  presentations,  contact  juiescarborough@niu.edu) 
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Assessment  and  the  Improvement  of 
Undergraduate  Education 

■ Quality  begins  with  an  organization 
culture  values: 

1.  High  expectations 

2.  Respect  for  diverse  talents  and  learning  styles 

3.  Emphasis  on  the  early  years  of  study 

■ A quality  curriculum  requires 

■ coherence  in  learning: 

4.  Synthesizing  experiences 

5.  Ongoing  practice  of  learned  skills 

6.  I ntegration  of  education  and  experience 


Jule  Scarborough,  2006 


Assessment  and  the 
Improvement  of  Undergraduate 

Education 

■ Quality  instruction  builds  in; 

8.  Active  learning 

9.  Assessment  and  prompt  feedback 

10.  Collaboration 

11. Adequate  time  on  task 

12. Out-of-class  contact  with  faculty 

(Education  Commission  of  the  States,  1995,  1996) 


Jule  Scarborough,  2006 


9 Principles  of  Good  Practice  for  Assessing 

Student  Learning 

1.  The  assessment  of  student  learning  begins  with 
educationai  values. 


2.  Assessment  Is  most  effective  when  It  reflects  an 
understanding  of  learning  as  muiti-dimensional, 
integrated,  and  revealed  in  performance  over  time. 

3.  Assessment  works  best  when  the  program  it  seeks  to 
improve  has  ciear,  expiicitiy  stated  purposes. 

4.  Assessment  requires  attention  to  outcomes,  but  aiso, 
and  equaiiy  to  the  experiences  that  lead  to  those 
outcomes. 


5.  Assessment  works  best  when  it  is  ongoing,  not  episodic 


Jule  Scarborough,  2006 


(Education  Commission  of  the  States,  1995, 1996) 


9 Principles  of  Good  Practice  for  Assessing 

Student  Learning 


6.  Assessment  fosters  wider  improvement  when 
representatives  from  across  the  educational 
community  are  involved. 

7.  Assessment  makes  a difference  when  it  begins 
with  issues  of  use  and  illuminates  questions  that 
people  really  care  about. 

8.  Assessment  is  most  likely  to  lead  to 
improvement  when  it  is  part  of  a larger  set  of 
conditions  that  promote  change. 

9.  Through  assessment,  educators  meet 

, , responsibilities  to  students  and  to  the  public. 

Jule  Scarborougn,  2006  ■ 


What  is  Assessment? 

Process  of  gathering  information  about 
what  students  know  or  can  do... 

Key  Question:  How  can  we  find  out  what 
students  are  learning? 

• We  can  observe  students  as  they  learn 

• We  can  examine  things  that  students  produce 


Jule  Scarborough,  2006 


Why  Do  Assessment? 


• Determine  student  accomplishments 

• Gather  data  for  evaluation  (i.e.  grades) 

• Improve  instruction 

• Address  demand  for  accountability 


Jule  Scarborough,  2006 


Balanced  Assessment 


• Uses  a variety  of  assessment  tools  consisting  of 
traditional  “tests”  and  newer  “authentic” 
performance  oriented  approaches 

• Each  assessment  targets  specific  features  that  are 
relevant  to  the  total  education  program 

• Assessments  are  “balanced”  in  such  a way  that 
the  strengths  of  one  offset  the  limitations  of 
another 


Jule  Scarborough,  2006 


Balanced  Assessment 


• Traditional  Assessment 

-Teacher-made  tests  for  grades  and  ranking 

• Portfolio  Assessment 

- Collection  of  student  work  for  growth  and 
development 

• Performance  Assessment 

- Authentic  assessment  of  final  products  or 
performances 


Jule  Scarborough,  2006 


Fig.  12.1  Multifaceted  Assessment  System 


Jule  Scarborough,  2006 


Kuhs  et  al  (2001)  p.  157 
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Traditional  Assessment 


True-False 

Matching 

Multiple-Choice 

Completion 


Essay 


Jule  Scarborough,  2006 


Portfolio  Assessment 


Self-Assessment  and  monitoring  of  learning 
Document  attainment  of  standards 
Chronicle  student  growth  and  development 
Allows  integration  across  subject  areas 


Provides  a communication  vehicle  between 
students,  professors,  and  others 


Jule  Scarborough,  2006 


Performance  Assessment 

• Mirrors  “real  life”  activities  outside  the  classroom, 
in  the  work  worid... Communities  of  Practice 

• Is  worthwhiie,  significant,  and  meaningful  to 
students,  because  it  is  reievant 

• Looks  and  feels  like  iearninq  activities  rather  than 
tests-not  artificial,  time-out... because  it  is 
integrated  and  inherent  within  the  iearning 
process 

• Invoives  higher-order  thinking  skilis  m 


Jule  Scarborough,  2006 


How  does  Assessment  differ 

from  Evaluation? 


Key  Point:  Evaluation  is  the  process  of 
interpreting  and  making  vaiue  judgments  about 
assessment  information,  resuiting  in: 

- teacher  evaiuation 

- curricular  evaluation 

- instructional  evaluation 


- students  grades 


Jule  Scarborough,  2006 


Assessment 


Purpose:  On-going  feedback  for 
improving  learning  and  instruction 

Evaluation 

Purpose:  Summative Judgment 

Terminal  Judgement 


Jule  Scarborough,  2006 


H*”-- 


ri  Ir 


■ 

r 


A\  -4: 


smen 


Uo  we  want  to  assess? 


_r 


j o w 


tio  we  want  to  assess? 


WHAT  type  evl 
desired? 


denoe  of  learning  is 


WHAT  do  grades  mean? 


Jule  Scarborough,  2006 


Through  learning  we  re-create  ourselves. 

Through  learning  we  become  able  to  do 
something  we  never  were  able  to  do  [before]. 

Through  learning  we  extend  our  capacity  to 
create,  to  be  part  of  the  generative*  process  of 
life. 


(Senge,  1990) 


LearninaBMMiwirsvnonvmous  with 
"taking  in  information" 


^lule  Scarborough,  2006 


Learning  = Shift  of  Mind 

" From  seeing  ourselves  ... 


e from  the  world 


■ to...C 
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(Senge,  1990,  2000) 
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(Senge,  1990) 


Transformative  Learning 


Grants  the  power  ... 


Relate  to  the  subject  matter 


iTo  Build  upon  existing  knowledge 


■To  Construct  new  knowledge,  and 


• Empowers  one  ... 


To  Create  their  desired  future 


WHAT  do  we  want  to  assess? 


■ Disciplinary  Knowledge  (K) 

■ Disciplinary  Skills  {S) 

m Disciplinary  Abilities  (A) 

■ I nterdisciplinary  KSAs 

■ Meta- cognitive  skills  & processes 

■ General  Education  knowledge  & skills 

■ Critical  literacies 

■ Transfer  of  KSAs  across  contexts 

■ Accreditation  standards 

■ I ntelligences 

■ Multi-cultural  capabilities 


Jule  Scarborough.  2006 


General  Education  Goals 


a.i.  communicate  in  writing 

a.ii.  communicate  by  speaking  and  listening 

a.iii.  quantitative  reasoning 

a. iv.  use  of  resources,  including  modern  technology 

b. i.  historical  development  of  culture 
b.ii.  value  of  the  arts 

b. iii.  science  and  social  science  methods 

c.  interrelatedness  of  disciplines 

d.  (global  awareness,  environmental  sensitivity, 

appreciation  of  cultural  diversity)  Jule  Scarborough,  2006 
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IShould  have  always 
been: 

* Social 

* Cultural 
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U L 


Intelligences 


Visual/  Spatial:  ■ Bodily/  Kinsethetic: 

Show  Me!  Just  Do  It! 

Logical/  Mathemati-  * Social: 

ca:  Why  Bother?  Can  We  Talk? 

Verbal/  Linguistic:  ■ * ntrapersonal/ 

Who  Says?  ■ I ntrospective: 

■ What's  in  It  for  Me? 


Musical/  Rhythmic:  ■ Naturalistic/  Physical 
I Meant!  World 


Disciplinary  KSAs 

ABET  Engineering 
ABET-Engineering  Technoiogy 
NAIT-  Industriai  Technoiogy 
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Standards  of  Assessment 

Quality 

Standard  1. 

• Quality  assessments  arise  from  and 
accurately  reflect  clearly  specified 
and  appropriate  achievement 
expectations  for  students 

Standard  2. 

• Sound  assessments  are  specifically 
designed  to  serve  instructional 
purposes 


Standards  of  Assessment 

Quality 

Standard  3. 

• Quality  assessments  accurately  reflect  the 
intended  target  and  serve  the  intended 
purpose 

Standard  4. 

• Quality  assessments  provide  a 
representative  sample  of  student 
performance  that  is  sufficient  in  its  scope 
to  permit  confident  conclusions  about 
student  achievement 


Standards  of  Assessment 

Quality 


■ Standard  5. 

• Sound  assessments  are  designed, 
developed,  and  used  in  such  a 
manner  as  to  eliminate  sources  of 
bias  or  distortion  that  interfere  with 
the  accuracy  of  results 


Examples  of  Assessment  Users  and 

Uses 


C J sj  5 J 


Assessment  User: 


Sample  Questions 

• Am  I succeeding? 

• Am  I improving  overtime? 

• Do  i know  what  it  means  to  succeed  here? 

• What  shouid  i do  next  to  succeed? 

• What  heip  do  I need  to  succeed? 

• Do  i feei  in  controi  of  my  own  success? 

• Does  my  teacher  think  i 'm  capabie  of  success? 

• Do  I think  i 'm  capabie  of  success? 

• I s the  iearning  worth  the  effort? 

• How  am  I doing  in  reiation  to  my  ciassmates? 

• Where  do  i want  aii  of  this  to  take  me? 


Assessment  User: 

■ Sample  Questions 

Are  my  students  improving? 

i s it  because  of  me? 

What  does  this  student  need? 

i s this  student  capabie  of  iearning  this? 

What  do  these  students  need? 

What  are  their  strengths  that  we  can  buiid  on? 

How  shouid  I group  my  students? 

Am  i going  too  fast,  too  siow,  too  far,  not  far 
enough? 

Am  i improving  as  a teacher 
How  can  i improve? 

Did  that  teaching  strategy  work? 

What  grade  do  I report? 


Fig.  3.3  Clarifying  Content  Priorities 


Fig.  6.2  Examples  of  Overarching  & Topical  Understandirn 


Overarcijfrig:Uncierstanclings; . 


• A presiideiTC  is  not  abov/e  the  law. 


• Democracy  rec^uires  a courageous,  not 
Just  a free,  press. 


The  motd&rn  novel  overturns  many 
tra<ditional  story  elements  an<d  nor-rY\& 
to  tell  a more  authentic  an<d  engaging 
narrative. 


CSravity  is  not  a physical  thing  but  a 
term  describing  the  constant  rate  of 
acceleration  of  all  falling  objects,  as 
found  through  experiment. 


FiJStulates  are  logically  prior  in  any 
axiomatic  system  but  developed  a-ftier 
the  fact  to  Justify  key  theorems.  They 
are  neither  true  nor  self-evident,  yet 
they  are  not  arbitrary. 


. TopIPigf  U naerstandipas  ir 


Watergate  was  a major  consti- 
tutional Crisis,  not  a “third-rate 
burglary”  (as  a Nixon  staffer  put 
it)  Or  mere  election  shenanigans 
between  political  part-ies. 


Holden  Caulfield  is  an  alienated 
antihero,  not  an  average  kid  on 
an  “excellent  adventure.” 


Vertical  height,  not  the  angle  and 
distance  of  descent,  determines 
the  eventual  “splashdown”  speed 
of  a falling  spacecraft- 


The  parallel  postulate  is  a crucial 
foundation  to  Euclidean  geometry, 
despite  its  awkwardness  and 
theoremlike  nature. 


• In  a free  market  ecov\orr\y,  price  is  a 
function  of  demand  versus  supply. 


Wiggins  & McTighe 
(2005)  p.  131 


» A baseball  card’s  worth  depends 
on  who  wants  it,  not  Just  its  condi- 
tion or  the  number  of  similar  ones 
available. 

• Sales  figures  from  eBay  reveal 
that  one  person’s  junk  is  another 
person’s  treasure. 


• Increased  scoring  Opportunities  in 
certain  sports  result  from  Creating 
space  on  offense  in  Order  to  spread 
the  defense  and  get  players  “open.” 


■*  Creating  space  and  exploiting  its 
creation  is  the  key  to  winning 
soccer. 

• The  defense  in  soccer  needs  to 
prevent  the  offensive  players 
from  getting  open  in  the  middle 
of  the  field. 


Jule  Scarborough,  200r 


Fig.  7.2  The  Logic  of  Backward  Design 
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If  the  desired  result  is  for  learners  to  ..  . C 

Then  you  need  evidence  of  the  student's  ability  to  . . . 

Meet  the  standards  . . . 

Standard  6 — Students  will  understand  essential  concepts  about 
nutrition  and  diet. 

6a — Students  will  use  an  understanding  of  nutrition  to  plan 
appropriate  diets  for  themselves  and  others. 

6c — Students  will  understand  their  own  eating  patterns  and 
ways  in  which  those  patterns  may  be  improved. 

Undetstand  that . . . 

• A balanced  diet  contributes  to  physical  and  mental  heath. 

• The  USPA  food  pyramid  presents  relative  guidelines  for  nutrition. 

• Pietary  rec^uirements  vary  for  individuals  based  on  age,  activity  level, 
weight,  and  overall  health. 

• Healthful  living  rec^ulres  an  Individual  to  act  on  available  Information 
about  good  nutrition  even  if  it  means  breaking  comfortable  habits. 

Thoughtfully  consider  the  questions  . . . ^ 

• What  is  healthful  eating? 

• Are  you  a healthful  eater?  How  would  you  know'? 

• How  could  a healthy  diet  for  one  person  be  unhealthy  for  another? 

• Why  are  there  so  many  health  problems  in  the  United  States  caused 
by  poor  eating  despite  all  the  available  information? 

Know  and  be  able  to  ..  . 

• Use  key  terms — protein,  fat,  calorie,  carbohydrate,  cholesterol. 

• Identify  types  of  foods  in  each  food  group  and  their  nutritional  values. 

• 3e  conversant  with  the  USPA  food  pyramid  guidelines. 

• Piscuss  variables  influencing  nutritional  needs. 

• Identify  specific  health  problems  caused  by  poor  nutrition. 

• Flan  a diet  for  different  kinds  of  people  in  different  kinds- of 
settings. 

• Feveal  an  understanding  that  the  USPA  guidelines  are  not 
absolute,  but  “guides” — and  that  there  are  other  guides  (as 
well  as  contextual  variables). 

• Carefully  note  and  analyze  the  habits  of  others  as  well  as 
oneself,  and  make  supported  inferences  about  why  people  eat 
the  way  they  do. 

That  suggests  the  need  for  specific  tasks  or  tests  like  . . . 

• Planning  meals  for  diverse  groups. 

• Reacting  to  excessively  rigid  or  loose  dietary  plans  made  by 

others.  | 

• Making  a good  survey  of  what  people  actually  eat  and  why. 

Quizzes.-  On  the  food  groups  and  the  USPA  food  pyramid 

Trompte:  Pescribe  health  problems  that  could  arise  as  a result  of 
, poor  nutrition  and  explain  how  these  could  be  avoided;  reflections  on 

one’s  own  eating  habits  and  those  of  others. 

Jule  Scarborough,  2006 


Wiggins  & McTighe  (2005)  p.  149 


Fig.  7.3  Two  Approaches  to  Thinking  About  Assessment 


When  thinking  like  WherV'thinking  like  an 

an  asse^or,;  we  ask— ^ activitx-jd^Ignef'Jjonty)#  we  ask—  - 


What,  would  be  sufficient,  and  revealing 
evidence  of  understanding? 

Owen  the  ^oals,  what  performance 
tasks  must  anchor  the  unit  and 
focus  the  instructional  work? 

What  are  the  different  types  of 
evidence  rec^uired  by  Sta^e  1 desired 
results? 

Against  what  criteria  will  we 
appropriately  consider  work  and 
assess  lev/els  of  equality? 

Did  the  assessments  reveal  and  dis-^ 
tin^uish  those  who  really  understood 
from  those  who  only  seemed  to?  Am 
i clear  on  the  reasons  behind  learner 
mista  kes? 


• What  would  be  fun  and  interesting 
activities  on  this  topic? 

• What  projects  might  students  wish 
to  do  on  this  topic? 

• What  tests  should  I give,  based  on 
the  content  I taught? 

'i 

How  will  I give  students  a grade 
(and  Justify  it  to  their  parents)? 

• How  well  did  the  activities  work? 

• How  did  students  do  on  the  test? 


Jule  Scarborough,  2006 


Wiggins  & McTighe  (2005)  p.  151 


Fig.  7.4  A Continuum  of  Assessments 


Jule  Scarborough,  2006 


Wiggins  & McTighe  (2005)  p.l52 


Fig.  7.5  Types  of  Evidence 


Performance  Tasks 

Complex  challenges  that  mirror  the  issues  and  problems  faced  by  adults.  Ranging  in  length  from  short-term 
tasks  to  long-term,  multistaged  projects,  they  yield  one  or  more  tangible  products  and  ’performances.  They 
differ  from  academ\c  prompts  in  the  following  ways: 

• Inv/olve  a real  or  simulated  setting  and  the  kind  of  constraints,  background  “noise,”  incenti^^es,  and 
opportunities  an  adult  would  find  in  a similar  situation  (i.e.,  they  are  authentic) 

• Typically  rec^uirc  the  student  to  address  an  identified  audience  (real  or  simulated) 

• Are  based  on  a specific  purpose  that  relates  to  the  audience 

• Allow  students  greater  opportunity  to  personalize  the  task 

• Are  not  secure:  The  task,  evaluative  criteria,  and  performance  sX-andards  are  known  in  advance  and 
guide  student  work 

Academic  Prompts 

Open-ended  questions  or  problems  that  reguire  the  student  to  think  critically,  not  just  recall  knowledge, 
and  to  prepare  a specific  academ\c  response,  product,  or  performance.  Such  guestions  or  problems 

• Reguire  constructed  responses  to  specific  prompts  under  school  and  exam  conditions 

• Are  “open,”  with  no  single  best  answer  Or  strategy  expected  for  solving  them 

• Are  often  "ill  structured,”  reguiring  the  development  of  a strategy 

• Involve  analysis,  synthesis,  and  evaluation 

• Typically  reguire  an  explanation  or  defense  of  the  answer  given  and  methods  used 

• Reguire  judgment-based  scoring  based  on  criteria  and  performance  standards 

• May  or  may  not  be  secure 

• Involve  guestions  typically  only  asked  of  students  in  school 

Quiz  and  Test  Items 

Familiar  a&aessmenX  formats  consisting  of  simple,  content-focused  items  that 

• Assess  for  factual  information,  concepts,  and  discrete  skill 

• Use  selected-response  (e.g.,  multiple-choice,  true-false,  matching)  or  short-answer  formats 

• Are  convergent,  typfcally  having  a single,  best  answer 

• May  be  easily  scored  using  an  answer  key  or  machine 

• Are  typically  secure  (i.e.,  items  are  not  known  in  advance) 

Informal  Checks  for  Understanding 

Ongoing  assessments  used  as  part  of  the  instructional  process.  Examples  include  teacher  guestioning, 
observations,  examining  student  work,  and  think-alouds.  These  assessments  provide  feedback  to  the 
teacher  and  the  student.  They  are  not  typically  scored  or  graded. 
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Fig.  7.11  Curricular  Priorities  & Assessment  Methods 


In  e^ecX-'wG  assessmen'ts.  we  see  a ma'tch  bel^ween  "the  "type  or  formal:-  of  "the  assessmerrC-  ar\eX  "C-he  nee<ded 
evidence  of  achieving  the  desired  results.  If  the  goal  i&  for  students  to  learn  basic  facts  and  akills.  then  paper- 
and-pencil  tests  and  c^uizzes  generally  provide  adeg^uate  and  efficient  measures.  However,  when  the  goal  is 
deep  understanding,  we  rely  on  more  complex  performances  to  determine  whether  our  goal  has  been  reached. 
The  graphic  below  reveal©  the  general  relationship  between  assessment  types  and  the  evidence  they  provide 
for  different  curriculum  targets. 
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Fig.  12.1  Multifaceted  Assessment  System 
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Figure  12.6  Traditional  Assessment  Pyramid  (Marzano  et  al.,  1993,  pp.  26-27) 


Figure  12.7  Reconfigured  Assessment  Pyramid  (Marzano  et  al.,  1993,  pp.  26-27) 
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Table  12.S  Features  of  Assessment  of,  for,  and  as  Learning 
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TABLE  2.2  Aligning  Achievement  Targets  to  Assessment  Methods 


Target  to 
Be  Assessed 

Assessment  Method 

Selected  Response 

Essay 

Performance 

Assessment 

Personal 

Communication 

Knowledge 

Mastery 

Multiple  choice, 
true/false,  matching, 
and  fill-in  can  sample 
mastery  of  elements 
of  knowledge 

Essay  exercises  can  tap 
understanding  of 
relationships  among 
elements  of 
knowledge 

Not  a good  choice 
for  this  target — ^three 
other  options 
preferred 

Can  ask  questions, 
evaluate  answers,  and 
infer  mastery — but  a 
time-consuming  option 

Reasoning 

Proficiency 

Can  assess 

understanding  of  basic 
patterns  of  reasoning 

Written  descriptions 
of  complex  problem 
solutions  can  provide 
a window  into 
reasoning  proficiency 

Can  watch  students 
solve  some  problems 
and  infer  about 
reasoning  proficiency 

Can  ask  student  to 
“think  aloud”  or  can 
ask  follow-up  questions 
to  probe  reasoning 

Skills 

Can  assess  mastery  of 
the  prerequisites  of 
skillful  performance — 
but  cannot  tap  the 
skill  itself 

Can  assess  mastery  of 
the  prerequisites  of 
skillful  performance — 
but  cannot  tap  the 
skill  itself 

Can  observe  and 
evaluate  skills  as  they 
are  being  performed 

Strong  match  when 
skill  is  oral 
communication 
proficiency;  also  can 
assess  mastery  of 
knowledge  prerequisite 
to  skillful  performance 

Ability  to 
Create  Products 

Can  assess  mastery  of 
of  knowledge 
prerequisite  to  the 
ability  to  create 
quality  products — but 
cannot  assess  the 
quality  of  products 
themselves 

Can  assess  mastery 
of  knowledge 
prerequisite  to  the 
ability  to  create 
quality  products — but 
cannot  assess  the 
quality  of  products 
themselves 

A strong  match  can 
assess;  (a)  proficiency 
in  carrying  out  steps 
in  product 
development  and 
(b)  attributes  of  the 
product  itself 

Can  probe  procedural 
knowledge  and 
knowledge  of  attributes 
of  quality  products — 
but  not  product  quality 
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lABLH/  4.0  AQva mages  ana  i/if»auvaiiLages  iu  Assessiueiii  ivieuiiKis 


^.ssessment  Mctliod. 

Description 

Advantages 

Disadvantages 

La  VVrillen  structured- 

Usually  timed,  fixed  or 

■ More  content  can  be  sampled  in 

B Guessing  is  a source  of  error 

response  assessments 

selected  response,  writ- 

short  testing  time 

■ Dilficull  to  construct  technically^ 

ten  exercise.s 

■ Can  measure  a range  of  cognitive 

etfective  items 

skills/bchiiviors 

B Cannot  be  adapted  to  measure 

■ Objcciivc  to  score 

some  behaviors  (social  behavio‘'r® 

B Efficient  to  score 

pnvedural  knowledge)  ^ 

■ Can  be  designed  to  have  high  in- 

■ Only  one.  correct  answer  possibje. 

lemal  consistency 

a Items  typically  measure  discrete^ 

■ Elficicnl  to  administer  to  large 

concepts  rather  than  inlcgraicJ 

groups 

knowledge  and  skills 

B Partial  credit  cannot  be  given 

lb  Written  open- 

Usually  timed,  con- 

■ Can  mcasLue  complex  thinking 

B Less  content  can  be  sampled, 

mded  assessments 

structedi  response,  writ- 

and  processes  (organization. 

given  testing  lime  - 

ten  exercises 

problem-solving,  creativity. 

■ Bluffing  causes  error 

integrated  skills) 

B Subjective  lo  score  without  clear.^ 

■ Partial  credit  can  be  given  to  an- 

criteria 

swers 

■ .Scorer  consistency  needed 

■ Allows  analytic  or  holistic 

B Time  consuming  to  score 

scoring 

B Cannot  be  used  for  nonwriiers 

B More  than  one  correct  answer 

B Human  scorers  needed 

pos.sihle 

B Scorer  training  may  bo  needed  ^ 

■ Efficient  to  administer  to  large 

■ 

groups 

1 Behavior-based 

Behaviors  or  dcnionsii  a- 

■ Can  measure  complex  perfor- 

B Time  consuming  to  make  nbser^.a- 

issessments 

tions  exhibited  in  natural 

■nianccs  and  behaviors  in  group 

tions 

or  structured  setting.s 

and  individual  settings 

B Time  consuming  to  score 

m Requires  actual  demonstration 

fl  Subjective  lo  scxsie  without  cleai^ 

and  direct  observation 

criteria  ^ ‘ 

■ More  than  one  coitccI  response 

B Less  content  can  be  sampled  iiL^ 

possible 

given  time 

■ Allows  analytic  or  holistic 

B Scorer  consistency  needed  ’ 
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Scorer  training  may  he  needed.^ '' 


n Ilf  nu.'  I 


a ^Tannnf  ht’  rfnnf?  wifh  Jarnn^^n/ifi 


? Product-based  I’rtxluCLs,  reports,  or 

issessmcnts  items  created  m struc- 

tured or  unstructured  sit- 
uations 


4 Interview-based  Onc-on-onc  verbal  (oral) 

assessments  interaction  in  structured 

or  unstructured  situa- 
tions 
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■ Can  measure  complex  thinking 
and  processes  (ot^ani^iatioii.  prob- 
lem-soKing,  creativity) 

■ Partial  credit  can  be  given  to  an- 
swers 

■ Allows  analvtic  or  holistic 
scoring 

■ More  than  one  coirect  answer 
possible 

■ Efficient  to  administer  to  large 
groups 

■ Requires  actual  production 

■ Requires  application  of  product 
develo|imeiil  skills 

■ Can  he  uniiiiicd  take-home  exer- 
cises 


■ Needed  when  oral  communica- 
tion is  the  focus  (hinguagc 
learning) 

■ Preferred  fonnai  for  very  young 
or  special  populations 

■ Cun  measure  a range  of  cogni- 
tive, personal -social,  and  eflec- 
tive  behaviors 

■ More  than  one  correct  answer 
possible 

■ Partial  credit  possible 

■ Allows  analytical  or  holistic 

trnnno 


■ Time  cons  uni  ing  to  score 

■ Less  content  can  be  sampled  in^ 
given  time 

m Limited  to  processes  that  result^? 
products 

Q Subjective  to  score  without  cleSf 
criicriii 

■ Scorer  consistency  needed 

■ Human  scorers  needed 

■ Respondents  can  get  outside 
unless  stmetnred  tightly 

■ Scorer  training  may  be  needed  - 

IV 


■ Time  consuming  to  administer 

■ Time  consuming  to  score 

■ Less  content  etui  be  sampled 
given  lime 

■ Limited  toproce^sses  that  need  ^ 
man  interaction 

■ Human  interviewers  needed 

■ Interviewer  training  and  slructd^ 
may  be  needed 

■ Subjective  to  score  withi-iut  clc^ 
critcri.i 

■ Scorer  consistency  needed 

Hnm'111  ('nrpre 


-!.»  uonnnucfi 


Assessment  Method  Description  Ad  vantages  Disadvuntages 


Portt'olios 

Purposeful  collections  of 

■ Can  be  ilesigncd  to  measure 

■ Requires  proper  coittcnl  sampling 

behaviors  of  work  sarn- 

growth  over  lime  or  periodic, 

■ Requires  adequate  number  of  sam- 

j)ies made  over  lime 

summativc  evaluations 

ples 

■ Can  be  easily  integrated  with  iii- 

B Limited  to  processes  or  bohaviqps 

siruction 

that  ciin  be  included  in  ti  port- 

■  Can  measure  complex  outcomes 

folio 

or  behaviors 

■ Subjective  to  score  willioui  clear' 

, 

■ Possible  10  indivkiiialize  or  ada(>t 

criteria 

to  groups 

■ Scorer  consistency  needed 

■ Can  measure  complex  (hinking 

■ Human  scorers  needed 

and  processes  (orgajiizalion,  prob- 

■  Respondents  can  gel  outside  help 

lem-solving.  crcaii\ily) 

■ Scorer  training  ntay  be  needed 

■ Partial  credit  possible 

' 

•j 

■ Allows  analytic  or  holistic 
scoring 

■ More  than  one  correct  answer 
possible 

■ Efllcieiii  to  administer  to  large 
groups 

■ Requires  production  of  actual 
samples 

■ Requires  application  of  product 
developmoni  skills 

■ Can  be  umimed  and  have  take- 

M 

liomc  exercises 
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Qu  izzes 


" Some  teachers  advocate  "surprise"  or  "pop" 
quizzes.  Their  reasoning  is  often  some  vaaue 
notion  that  a student  should  always  be 
prepared  to  perform  on  command.  This  seems 
to  be  an  unrealistic  expectation  of  students. 

Teachers,  for  example,  make  lesson  plans  and 
prepare  to  teach  these  lessons  in  advance. 

They  are  often  resentful  (and  rightfully  so)  if 
asked  to  teach  a class  for  which  they  have  not 
had  sufficient  time  to  prepare,  ,2ooi,p.3ii) 


HOW  do  we  want  to  assess? 


Operational  Philosophy; 

• multiple  opportunities 

• diverse  assessment  procedures 

How 


Traditional  testing 
Probiem-based  ^ 
Performance- based 


^ Problem-based 


WfJ^VJi 


type  of  evidence  of  learning 
is  desired? 


■ I ndicators  of  performance 

■ Performances 

■ Transfer  across  contexts 
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Make  judgments 
r about  the  value  of 
ideas  or  materials. 


SYNTHESIS 

Builds  a structure  or  pattern  from 
diverse  elements.  Put  parts  together 
to  form  a whole,  with  emphasis  on 
creating  a new  meaning  or  structure 


XT 


ALYSIS 


Separates  material  or  concepts  into  component  parts  so 
that  its  organizational  structure  may  be  understood. 
Distinguishes  between  facts  and  inferences. 


LIGATION 

Use  a concept  in  a new  situation  or  unprompted  use  of  an 
abstraction.  Applies  what  was  learned  in  the  classroom  into 
novel  situations  in  the  workplace. 


^ COMPREHENSION 

Builds  a structure  or  pattern  from  diverse  elements.  Put  parts  together  to  form  a 
whole,  with  emphasis  on  creating  a new  meaning  or  structure. 


KNOWLEDGE 

Recall  of  data. 
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Valued  Activities/Authenticity 


■ Test  items  - meant  (only)  to  be 
useful  as  indicators  of  valued 
"real-world"  performances 
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Leadership 


PR&D 


Performance  Assessment 


Emphasizes  the  identification  and  solution 
of  real-world  problems  using  reasoning 
and  higher-order  thinking  skills 

Based  upon  a community  of  practice 


learning  in  situ 
with/from  each  other 
situated  in  action 

in  the  community  of 
practice 
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Characteristics  of  Performance 

Assessment 

Open-ended  tasks 

Higher-order,  complex  skills 

Extended  periods  of  time  for 
performance 

Group  performance 

Student  and  teacher  choice  of 
tasks 

J udgment  scoring 
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Example  Performance  Task 

Information  Processing: 

Select  a product  which  is  readily  available  in 
local  stores  and  can  be  tested  in  the  science 
lab.  Collect  information  on  the  product  from 
wide  variety  of  sources— magazines, 
newspapers,  internet,  and  a sample  of  people 
who  use  the  product.  Keep  a list  of  those 
sources  and  be  prepared  to  report  on  which 
information  was  the  most  relevant  and  which 
was  not  very  useful. 


Example  Performance  Task 


Habits  of  mind: 

Establish  a procedure  for  testing  your 
product  prior  to  beginning  the  project. 
Write  down  the  procedure  in  a step-by-step 
order.  Make  sure  that  the  elements  of  the 
scientific  method  are  included. 


Example  Performance  Task 


Cooperation: 

Assign  specific  duties  to  each  member  of 
your  research  team.  Keep  a log  of  team 
activities  and  progress  of  your  testing.  As 
you  work  together,  you  will  find  that  you 
must  change  certain  things  to  make  your 
team  work  more  effectively.  Be  aware  of 
those  behaviors  you  had  to  change  and 
what  you  did  to  change  them. 
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Example  Performance  Task 


Communication : 

Present  your  conclusions  and  findings  in  one  of  the 

following  ways: 

- A written  research  report  including  data  tables, 
charts,  and  diagrams. 

- An  article  written  for  Consumer  Reports  magazine, 
complete  with  suggested  photos,  charts,  tables,  and 
diagrams. 

- An  oral  presentation  of  your  test  results  which 
incorporates  visual  aids.  Allow  listeners  to  ask 
questions  and  review  your  data  gathering  and 
analysis  techniques. 


Completed  Performance  Task 

You  are  a member  of  a team  of  scientists  who  are  charged  with  testing  the  quaiity  of  a 
consumer  product.  Your  team  must  design  an  experiment  to  test  the  product,  collect 
appropriate  data,  draw  conclusions,  and  compile  a report  regarding  the  quality  of  the 
product. 

Select  a product  which  is  readily  available  in  local  stores  and  can  be  tested  in 
the  science  lab.  Collect  information  on  the  product  from  a wide  variety  of 
sources--magazines,  newspapers,  internet,  and  a sample  of  people  who 
use  the  product.  Keep  a list  of  those  sources  and  be  prepared  to  report  on 
which  information  was  the  most  relevant  and  which  was  not  very  useful. 

Establish  a procedure  for  testing  your  product  prior  to  beginning  the  project. 
Write  down  the  procedure  in  a step-by-step  order.  Make  sure  that  the 
elements  of  the  scientific  method  are  included. 

Assign  specific  duties  to  each  member  of  your  research  team.  Keep  a log  of 
team  activities  and  progress  of  your  testing.  As  you  work  together,  you 
will  find  that  you  must  change  certain  things  to  make  your  team  work 
more  effectively.  Be  aware  of  those  behaviors  you  had  to  change  and 
what  you  did  to  change  them. 

Present  your  conclusions  and  findings  in  one  of  the  following  ways: 

• A written  research  report  including  data  tables,  charts,  and  diagrams. 

• An  article  written  for  Consumer  Reports  magazine,  complete  with 
suggested  photos,  charts,  tables,  and  diagrams. 

• An  oral  presentation  of  your  test  results  which  incorporates  visual  aids. 
Allow  listeners  to  ask  questions  and  review  your  data  gathering  and 
analysis  techniques. 


Check  For: 

Complex  thinking  - Critical  thinking 

(scenario  requires  compiex  thinking  skiiis) 

I nformation  processing 

(scenario  requires  gathering,  processing,  & evaiuating 
information) 

Habits  of  mind 

(scenario  requires  seif  reguiation  & criticai  thinking) 

Collaboration/  Cooperation  - Teamwork 
Tangible  product/  performance  - Real  World 


Figure  2.1  Key  Differences  Between  Typical  Tests  and  Authentic  Tasks 


Tifpical  Tests 

Authentic  Tasks 

Indicators  of  Authenticity 

Require  correct 
responses  only 

Require  quality  product  and/or 
performance,  and  /ustificatm. 

We  assess  whether  the  student  can 
explain,  apply,  self-adjust,  or  justify 
answers,  not  just  the  correctness 
of  answers  using  facts  and 
algorithms. 

Must  be  unknown 
in  advance  to  ensure 
validity 

Are  known  as  much  as  possible 
in  advance;  involve  excelling  at 
predictable  demanding  and 
core  tasks;  are  not  "gotcha!" 
experiences. 

The  tasks,  criteria,  and  standards 
by  which  work  will  be  judged  are 
predictable  or  known— like  the 
recital  piece,  the  play,  engine  to  be 
fixed,  proposal  to  a client,  etc. 

Figure  2.1  Key  Differences  Between  Typical  Tests  and  Authentic  Tasks 

Tijpkal  Tests 

Authentic  Tasks 

Are  disconnected  from 
a realistic  context  and 
realistic  constraints 

Require  real-world  use 
of  knowledge:  the  student 
must  "do"  history,  science, 
etc.  in  realistic  simulations 
or  actual  use. 

The  task  is  a challenge  and  a set  of 
constraints  that  are  authentic — 
likely  to  be  encountered  by  the 
professional,  citizen  or  consumer. 
(Know-how,  not  plugging  in,  is 
required.) 

Contain  isolated 
items  requiring  use  or 
recognition  of  known 
answers  or  skills 

Are  integrated  challenges  in 
which  knowledge  and  judgment 
must  be  innova tively  used  to 
fashion  a quality  product  or 
performance. 

The  task  is  multifaceted  and  non- 
routine, even  if  there  is  a "'right" 
answer.  It  thus  requires  problem 
clarification,  trial  and  error,  adjust- 
ments, adapting  to  the  case  or  facts 
at  hand,  etc. 

Are  simplified  so 
as  to  be  easy 
to  score  reliably 

Involve  complex  and  non- 
arbitrary  tasks,  criteria,  and 
standards. 

The  task  involves  the  important 
aspects  of  performance  and/or 
core  challenges  of  the  field  of  study, 
not  the  easily  scored;  does  not 
sacrifice  validity  for  reliability. 

J J 
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Figure  2.1  Key  Differences  Between  Typical  Tests  and  Authentic  Tasks 

Typical  Tests 

Autheittic  Tasks 

Ind ica tors  of  Aiithen t id ty 

Are  one  shot 

Are  iterative:  contain  recurring 
essential  tasks,  genres,  and 
standards. 

The  work  is  designed  to  reveal 
whether  the  student  has  achieved 
real  versus  pseudo  mastery,  or 
understanding  versus  mere 
familiarity,  over  time. 

Depend  on  higWy 
technical  correlations 

Provide  direct  evidence, 
involving  tasks  that  have  been 
validated  against  core  adult 
roles  and  discipline-based 
challenges. 

The  task  is  valid  and  fair  on  its 
face.  It  thus  evokes  student  interest 
and  persistence,  and  seems  apt  and 
challenging  to  students  and 
teachers. 

Provide  a score 

Provide  usable,  diagnostic 
(sometimes  concurrent) 
feedback:  the  student  is  able 
to  confirm  results  and  self-adjust 
as  needed. 

The  assessment  is  designed  not 
merely  to  audit  performance  but  to 
improve  future  performance.  The 
student  is  seen  as  the  primary 
"customer"  of  information. 

Jule  Scarborough,  2006 


Authentic  Assessment  involves: 


Tasks  which  look  and  feel  like 
learning  activities  rather  than  tests 


Tasks  which  are  worthwhile, 
significant,  and  meaningful 
(relevant)  to  students 

Tasks  which  involve  higher-order 
thinking  skills 


Authentic  Assessment  is  designed  to: 


stress  important  understandings  and  abilities 
be  educational  and  engaging 
be  a part  of  the  curriculum 
reflect  real-life,  interdisciplinary  chailenges 


present  students  with  complex,  open-ended 
problems  and  tasks  that  integrate  knowledge  and 
skills 


Authentic  Assessment  is  structured: 


with  tasks  which  warrant  practicing  and 
repeating 

so  that  coiiaboration  with  other  students  is  often 
necessary 

such  that  students  know  in  advance  how  they  wili 
be  assessed 


so  that  varying  amounts  of  time  are  availabie 
to  allow  for  a significant  degree  of  student  choice 


Definition 

Authentic  Performance  Assessment 


^ A description  of  a “reai  iife”  project  or 
task  in  which  students  simuitaneousiy 
iearn  and  are  assessed  according  to 
estabiished  benchmarks. 
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Authentic  Performance  Tasks 

■ I nvolve  engaging  problems  and  questions  of 
importance  and  substance 

■ Simulate  challenges  facing  persons  in  the  "real 
world" 

■ Require  a repertoire  of  knowledge  and  skills 

■ Focus  on  a student's  ability  to  produce  a product 
or  performance 

■ I nvolve  "de- mystified"  criteria  and  standards 


Authentic  Performance  Tasks 


Rely  on  trained  assessor  judgment 

Often  involve  interactions  between  assessor  and 
student 

Often  requires  the  student  to  Justify  answers  and 
respond  to  follow-up  questions 

Provides  chalienges  where  the  product 
determines  the  quality  of  the  resuit 

I nvolves  patterns  of  response  and  behavior 


Constructing 
Performance  Tasks 


Constructing  Performance  Tasks 


■ Step  1:  Identify  a content  standard  to  be 
included.  Standards  from  an  integrated 
module  might  include: 

Insert  ABET/TAC/NAIT  -standard  & SLO(s) 
here:  (Can  be  a "cluster”) 

• 5A:  Locate,  organize,  and  use  information  from 
various  sources  to  answer  questions,  solve  problems 
and  communicate  ideas. 

• 8B:  Interpret  and  describe  numerical  relationships 
using  tab  es,  graphs  and  symbols. 

• llA:  Know  and  apply  the  concepts,  principles  and 
processes  of  scientific  inquiry. 


Constructing  Performance 

Tasks 

Step  2:  Structure  the  task  around  a 
real  world  scenario  from  Community 
of  Practice  - real  and  relevant. 


“Real-World”  Scenario 


A paragraph  which  describes  a scenario 
in  the  world  outside  of  the  classroom  in 
communities  of  practice  where 
individuals'  jobs  require  them  to 
demonstrate  skills  and  knowledge  which 
are  addressed  in  the  selected  goal, 
standards,  and  benchmarks. 


Example  “Real-World”  Scenario 


You  are  a member  of  a team  of  scientists 
who  are  charged  with  testing  the  quality 

of  a consumer  product.  Your  team  must 
design  an  experiment  to  test  the  product, 
collect  appropriate  data,  draw  conclusions, 
and  compile  a report  regarding  the  quality 
of  the  product 


Scoring 

Performance 

Tasks 


Assessment  Problems  Associated 
with  Performance  Tasks 


There  is  no  single  correct  response 
Tasks  are  often  done  by  groups  of  students 
They  require  qualitative  judgements  by  teachers 
Grading  is  often  time  consuming 
Qualitative  judgements  may  be  inconsistent 


Solution 

Rubrics 


Rubrics  Contain: 


Established  set  of  criteria  used  in  scoring 
or  rating  student  performance 

The  criteria  take  the  form  of  performance 
descriptors  associated  with  predetermined 
levels  of  achievement 

Rubrics  may  be  supplemented  with  a 
benchmark  for  each  descriptor  which 
indicates  a sample  of  student  work  which 
meets  the  criteria. 


Table  12.6  Rubric  for  Assessing  the  Quality  of  a Performance  Task 

Key  Components 

A properly  designed  Performance  Task  must; 

a.  be  based  on  content  standards  taken  from  the  Illinois  Learning  Standards 

b.  describe  a “real-life”  scettario 

c.  involve  students  in  complex  reasoning  processes 

d.  require  students  to  collect  and  process  information 

e.  incorporate  “habits  of  mind” 

f require  student  collaboration  and  cooperation 

g.  result  in  a tangible  product  or  communication  activity 

I.  Component:  The  Performance  Task  is  based  on  the  Illinois  Learning  Standards. 

1.  The  Perforrrrance  Task  is  directly  related  to  and  based  on  Learning  Standards. 

2.  Learning  Starxlards  are  apparent,  but  the  relation  to  the  task  is  sketchy  or  irrelevant. 

3.  The  Performance  Task  does  not  appear  to  be  based  on  Learning  Standards. 

II.  Component;  A “Real-Life”  scenario  is  described  in  the  Performance  Task. 

1.  The  scenario  described  in  the  task  accurately  mirrors  an  activity  in  the  community 
outside  the  classroom. 

2.  The  scenario  described  in  the  task  simulates  an  activity  in  the  community  outside  the 
classroom. 

3.  The  scenario  described  in  the  task  contains  some  aspects  of  activity  outside  the 
classroom  but  is  largely  contrived. 

4.  The  scenario  described  in  the  task  is  an  academic  exercise  that  usually  takes  place 
only  in  the  context  of  a school  setting. 

III.  Component:  The  Performance  Task  involves  students  in  complex  reasoning  processes. 

1.  The  task  requires  students  to  utilize  complex  reasoning  components,  such  as 
induction/deduction,  diagnosis,  abstracting,  experimental  inquiry,  or  problem 
solving. 

2.  The  task  requires  students  to  utilize  complex  reasoning  components,  such  as 
comparing,  classifying,  decision  making,  or  investigation. 

3.  The  task  requires  students  to  only  recall  facts. 

IV.  Component:  The  Performance  Task  requires  students  to  collect  and  process  information. 

1.  The  task  incorporates  a variety  of  information  gathering  techniques  and  information 
resources.  Students  are  required  to  interpret  and  synthesize  information  and 
accurately  assess  the  value  of  information  gathered. 

2.  The  task  requires  students  to  gather  and  synthesize  information,  but  the  value  of  the 
information  gathered  is  not  assessed. 

3.  The  task  requires  students  to  gather  information  but  not  to  interpret  it 

4.  The  task  requires  no  gathering  or  processing  of  information. 
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Virtues  of  Rubrics 

■ They  communicate  key  components  of 
student  performance 

■ They  promote  easier  assessment 

■ They  increase  reiiabiiity  and  consistency 

■ They  show  students  where  they  are  and 
how  they  can  improve 

■ They  aid  in  documenting  progress 


Example  Rubric 

Key  Component: 

Drawing  conclusions  based  on  experimental  data 

Level 

■ 3 Draws  a conclusion  that  is  supported  by  the 

data  and  gives  supporting  evidence  for  the 
conclusion 

■ 2 Draws  a conclusion  that  is  supported  by 

data,  but  fails  to  show  any  evidence  for  the 

conclusion 

■ 1 Draws  a conclusion  that  is  not  supported 

by  data 

■ 0 Fails  to  reach  a conclusion 


Example  Performance  Task 

You  are  a member  of  a team  of  scientists  who  are  charged  with  testing  the  quality  of  a consumer 
product.  Your  team  must  design  an  experiment  to  test  the  product,  collect  appropriate  data, 
draw  conclusions,  and  compile  a report  regarding  the  quality  of  the  product. 

Select  a product  which  is  readily  available  in  local  stores  and  can  be  tested  in  the 
science  lab.  Collect  information  on  the  product  from  a wide  variety  of  sources-- 
magazines,  newspapers,  internet,  and  a sample  of  people  who  use  the  product. 
Keep  a list  of  those  sources  and  be  prepared  to  report  on  which  information  was 
the  most  relevant  and  which  was  not  very  useful. 

Establish  a procedure  for  testing  your  product  prior  to  beginning  the  project.  Write 
down  the  procedure  in  a step-by-step  order.  Make  sure  that  the  elements  of  the 
scientific  method  are  included. 

Assign  specific  duties  to  each  member  of  your  research  team.  Keep  a log  of  team 
activities  and  progress  of  your  testing.  As  you  work  together,  you  will  find  that  you 
must  change  certain  things  to  make  your  team  work  more  effectively.  Be  aware  of 
those  behaviors  you  had  to  change  and  what  you  did  to  change  them. 

Present  your  conclusions  and  findings  in  one  of  the  following  ways: 

• A written  research  report  including  data  tables,  charts,  and  diagrams. 

• An  article  written  for  Consumer  Reports  magazine,  complete  with  suggested 
photos,  charts,  tables,  and  diagrams. 

• An  oral  presentation  of  your  test  results  which  incorporates  visual  aids.  Allow 
listeners  to  ask  questions  and  review  your  data  gathering  and  analysis 
technioues. 


Identify  Key  Components 


Product  Testing 

• Design  a test  procedure 

• Follow  the  procedure 

Data  Collection 
Drawing  Conclusions 


Reporting  Results 


Rubrics 


■ Reveals  the  scoring  'Vules" 

-explains  the  criteria  against  which 
student  work  will  be  judged 

■ More  importantly! 

-makes  public  key  criteria  that 
students  can  use  in  developing, 
judging,  and  revising  work 


Jule  Seal boi'ougi'i.  2006 


Figure  7.2  Rubric  for  Open-Ended  Mathematical  Problems 


Derucmstrnted  Competence 

Exeinplary  Response; 
Ratinj?  = 

Gives  a complete  response  with  a clear,  colrerent,  unambiguous,  and 
elegant  explanation;  includes  a clear  and  simplified  diagram; 
commujiicates  oHectively  to  trie  identified  audience;  shows 
understanding  of  the  problem's  mathematical  ideas  and  processes; 
identifies  all  the  important  elements  of  the  problem;  may  include 
examples  and  counterexamples;  presents  strong  supporting 
arguments. 

Coinpctent  Response: 
Rating  = 5 

Gives  a fairly  complete  response  with  reasonablv  clear  explanations, 
may  include  an  appropriate  diagrain;  communicates  effectively  to  the 
identified  audience;  shows  understanding  of  the  problem's  ideas  and 
processe.s;  identities  most  important  elements  of  the  problem; 
presents  solid  supporting  arguments. 

Sntisfncfo7-\/  Response 

Minor  Flaws  But 
Satisfactory; 
Raring  = 4- 

Completes  the  problem  .sari.sfactorily,  hut  the  explanation  mav  be 
muddled:  argumentation  may  be  incomplete;  diagram  maybe 
inappropriate  or  unclear;  understands  the  undcrrlying  mathematical 
ideas,  uses  ideas  effectively. 

Serious  Flaws  But 
Nearly  Satisfactory: 
Raring  — 3 

Begins  the  problem  appropriately  but  may  fail  to  complete  or  may 
omit  significant  parts  of  the  problem;  may  tail  to  show  full  under- 
standing of  mathemaricaJ  ideas  and  processes;  may  make  major 
computational  c'rrors;  may  misuse  or  fail  to  use  mathematical  terms; 
response  may  reflect  an  inappropriate  strategy  for  solving  the  problem. 

Jtiadeqiiate  Response 

Begins,  But  Fails  to 
Complete  Problem: 
Rating  = 2 

Explanation  is  not  understandable;  diagram  may  be  unclear;  shows 
no  understanding  i.>f  the  problem  .situation;  may  make  major 
computational  errors. 

Unable  tc>  Begin 
Effectively: 
Raring  - f 

W'ords  used  do  not  reflect  the  problem;  drawings  misrepresent 
the  problem  situation;  fails  to  indicate  which  information  Ls 
appropriate. 

No  Attempt 

Rating  = U 

Jule  Scarborough,  2006 


Appendix 


Figure  7,b  Kubrics  tor  a bocratic  beminar 


Excellent 


Good 


Conduct  Leadership  RniSiming  Lisleiiing  Reading 


Demonstrates  respect, 
entlmsiasm,  and  skill  for 
the  purpose  of  seminar; 
insight  into  important 
texts  and  ideas  gained 
tlirough  the  interplay 
of  collaborative  and  per- 
sonal inquiry  into  a text. 
Demonstrates  in  speech 
and  manner  a habitual 
respect  for  the  text,  rea- 
soned discussion,  and 
shared  inquiry.  Effectively 
contributes  to  deepening 
and  bniadBiing  the 
conversation,  revealhig 
exemplary  habits  of  mind 


Takes  clear  responsibility 
for  the  seminar's  progress 
or  lack  of  it  Takes  stock  of 
the  overall  direction  and 
eltectiveness  of  the  discus- 
sion, and  takes  apt  steps 
to  refocus  or  redirect 
conversation  and/or  to 
cause  others  to  rclliink 
previous  statements. 
Offers  apt  feedback  and 
effective  j^idance  to 
others.  Takes  steps  to 
invoh'C  reticent  partici- 
pants and  In  ensure  that 
umioticed  points  are 
attended  to. 


Arguments  are  reasonable, 
apt,  logical  and  substanti- 
ated with  evidence  from 
the  text  so  as  to  consisr 
tently  move  tlie  conversa- 
tion forward  and  deepen 
the  inquiry.  Die  analyses 
made  are  helpful  m 
clarifying  complex  ideas. 
Criticisms  made  are  never 
ad  hoininc’m. 


Listens  unusually  well. 
Takes  steps  routinely 
to  comprehend  what  is 
said,  is  consistently  atten- 
ti\'e  {as  reflected  in  direct 
and  indirect  evidence). 
Later  responses  {actions, 
comments,  and  writings) 
indicate  accurate  and 
peiceptiv'e  recall  of  what 
was  said  and  by  whom. 


Conduct  and  wnitten  work 
indicate  student  has  read 
the  text  carefully,  is  thor- 
oughly famihar  with  the 
text's  main  ideas,  can  oiler 
insightful  interpretatioiw 
and  evaluations  ot  it,  is 
respectful  of  the  text  while 
also  reading  It  critically, 
and  has  come  prepared 
with  thoughtful  questions 
and  reactions. 


Demonstrates  in  speech 
and  maimer  an  overall 
respect  for  and  under- 
standing of  the  goals, 
processes,  and  norms  of 
reasoned  discussion  and 
shared  mquiry.  Parhri- 
pates  to  advance  con- 
versation and  displays 
mature  liabits  of  mind, 
but  may  sometimes  be 
ineffective  in  slwmg  in- 
sights, advancing  inquiry, 
or  working  with  others. 


Is  generally  willing  to  take 
on  facihlative  roles  and 
responsibilides.  Either 
makes  regular  efforts  to 
be  helpful  (in  moving  the 
com'ersation  forward 
and/or  including  others 
in  It)  but  is  sometimes 
ineffective  in  doing  so; 
or  does  not  typically  take 
a leadership  role  but  is 
effective  wlien  does  so. 


Arguments  are  generally 
reasonable,  apt,  and 
logical.  There  may  be  some 
minor  flaws  in  reasoning, 
evidence,  or  aptness  of 
remarks,  but  the  ideas 
contribute  to  an  under- 
standmg  of  the  text  or  of 
comments  made  by  others. 
Criticisms  arc  rarely  ad 
hominem 


Listens  well.  Takes  steps  to 
comprehend  what  is  said 
Generally  pays  attention 
and/or  responds  appro- 
priately to  ideas  and 
questions  offered  by 
other  participants.  Later 
responses  involve  accurate 
recall  of  what  was  said 
and  by  whom. 


Conduct  and  written  work 
generally  indicate  student 
has  read  the  text  carefully, 
grasps  the  mam  ideas,  can 
offer  reasonable  (if  some- 
times incomplete  or  sur- 
face) interpretations,  and 
has  come  with  apt  ques- 
tions and  ideas  regarding 
the  text. 
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'Figure  7.6  Continued 

Condvct  Lendership 


Speech  and  manner 
suggest  that  the  student 
Tnisimderstands  the  pur- 
pose ol  the  disnissiun 
and/or  Ls  undisciplined 
concerning  seminar 
practices  and  liabitsof 
mind.  May  contribute, 
even  fre(|uenlly,  to  con- 
versation but  is  ineffective 
due  to  opinionated, 
unclear,  and/ or  inade- 
quately explicit  views-, 


Takes  on  facilitativc  roles 
and  responsibilities  infre- 
quentlv  and/ or  ineffec- 
tively. When  taking  on 
a leadership  role,  may  mis- 
construe the  responsibility 
by  lobbying  for  favored 
opinions  or  speakers  only, 
and/or  by  trying  to  close 
off  discussion  of  diverse 
and  unresolved  views  in 
favor  of  ncat-and<Iean 
premature  closure. 


Unsatisfactory 


Speech  and  manner  dis- 
play little  respect  for 
and/or  understanding 
ol  the  .seminar  process. 
Student  appears  to  lack 
essential  habits  of  muid: 
is  either  routinely  argu- 
mentative, distracting, 
and/or  ol-«tinate,  or  is 
disenga  ged— extremely 
reluctant  to  participate, 
even  wlien  called  upon  (to 
the  point  of  making  otliers 
.feel  tb.edetachmcnt)'.;. 


Plays  no  active  facilitation 
role  of  any  kind,  or  actions 
are  consistently  counter- 
productive in  that  role. 


Reasoning 


Listening 


Reading 


Unsubstantiated  or  un- 
developed opulions  are 
offered  more  than  sound 
arguments  Conunenta 
suggest  that  the  student 
has  some  difficiiltv  in 
mo\'ing  beyond  mem 
reactions  to  more  thorough 
arguments,  or  difficulty  m 
following  the  complex 
arguments  of  others  (as 
reflected  in  questions 
asked  and/or  non 
sequiturs)  Student  may 
sometimes  resort  to  ad 
honiinem  attacks  instead 
of  focusing  on  the  aitique 
of  clainis  and  arguments. 


Comnients  suggest  that 
student  lias  great  difficulty 
with  analytical  require- 
ments of  semin^ir.  Remarks 
routinely  apf.iear  to  be  non 
sequiturs  and/or  so 
illogical  or  without  sub- 
stantiation as  to  be  not 
followable  by  others. 
Student  may  often  resort 
to  ad  hominem  comments 
to  text  author  and  other 
students. 


Does  not  regularly  listen 
very  well  and/or  is  not 
always  attentive,  as 
reflected  in  comments  and 
body  language.  Verbal 
reactions  reflect  an  earlier 
difficulty  or  failure  to 
listen  carefully  to  what 
was  said.  Behavior  may 
signify  either  that  the 
student  lacks  effective 
note-taking  strategies 
and/or  does  not  gra.sp 
the  importance  of  listening 
to  different  points  of  view 
and  reflecting  on  them. 


Does  not  listen  adequately, 
as  reflected  in  la  ter  ques- 
tions or  conunents  (for 
example,  non  sequiturs 
and  repetition  of  earlier 
points  as  if  they  hadn’t 
been  spoken)  and/ or  body 
language  that  is  very  sug- 
gestive of  inattentiveness. 


Comments  indicate  that 
tire  student  may  have  rea^ 
the  text  but  has  either 
misunderstood  it  (due 
either  to  difii’culties  in 
reading  and/or  assuming 
a stance  that  is  too  ego- 
centric or  present- 
centered)  or  lias  not  put 
enough  disciplined  and 
focused  effort  into  pre- 
paring for  the  seminar. 
Varying  patterns  of  parti^ 
cipation  also  suggest  that 
the  student's  preparation 
is  inconsistent. 


Student  a tlier  is  generally 
unable  to  make  adequate 
meaning  of  texts  or  lias 
generally  come  to  class 
unprepared  The  student? 
may  be  unable  to  read 
complex  texts  and/or 
may  not  know  or  use 
disciplined  strategies  for 
understandmg  and  taking 
notes  on  such  texts. 
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^^UDric  torronnal  uraj  ^^ommurucauon  in  a oraauaie  rrogram 


Levels  of  Achievement 

Criteria 

3 

Sophisticated 

2 

Competent 

1 

Not  Yet  Competent 

Drgatiiznthn 

Presentation  is  cleniv  logical  and 
organized.  Listener  can  follow 
line  of  reasoning. 

Presentation  is  generally  dear 
and  rvcll  organized.  A few 
minor  points  may1)e  confusing 

Listener  can  follow'  presentatioj], 
only  with  effort.  Some  argume:^ 
are  not  dear.  Organization  see^ 
haphazard. 

Ityle 

n , 1 

t »*  >•  * 

Level  of  presentation  is  appropri- 
ate for  the  audience.  Presentation 
is  a planned  convei  sahon,  paced 
for  audience  Linderstanding.  It  is 
not  a reading  of  a paper.  Speaker 
is  dearly  comfortable  in  front  of 
the  group  and  wn  be  heard  by  all. 

Level  of  pi'esentation  is  generally 
appropriate.  Pacing  is  sometimes 
too  fast  Or  slow.  The  pre.si-nter 
seems  slightly  uncomfortable  at 
times,  and  tlie  audience  occasion- 
ally has  trouble  hearing  him/her 

Aspects  of  presentation  are  loc3 
elementary  or  too  sophisticated:: 
for  audience.  Presenter  seems 
uncomfortable  and  can  be  hear^ 
only  if  listener  is  very  attentivi^: 
Mudi  of  the  information  is  rea^. 

llstofCoitmtimcatm 
^ids  (e.g..  Tmispm'm’ies, 
Slidcit,  Posters,  JIaiidofits, 
Computer-Generated  Materials) 

%* 

Comnmnication  aids  enhance  the 
presentation  They  are  prepared 
in  a professional  nuinncr 
►•Font  on.visuals  is  large ,ci\ough 

U.ir  «11 

Communication  aids  contribute 
to  the  quality  of  the  presentation 
Font  size  is  appropriate  for  read- 
ing. Appropriate  inforiratipais 

n 1 1 i^'T  #^>4  f***  f 

Communication  aids  arc  poor^ 
prepared  or  used  inappropriately 
Font  is  too  small  to  be  easily 
Ipen.  Tqo  much  infprt|y|tiofi 

♦ VI 1 f r I’d 
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Content 


Depth  of  Content 

Speaker  pro\ddes  an  accurate  and 
complete  explanation  of  key 
concepts  and  theonesy  drawing 
upon  relevant  literature.  Applica- 
tions  of  theory  are  included  to 
illuminate  issues.  Listeners  gain 
insights. 

fVccuracy  of  Content 

Information  (names,  facts,  etc.) 

included  in  the  presentation  .is 
consistently  accurate. 

i, 

'isc  oflon^Hd^e 

Sentences  are  complete  and 
grammatical,  and  they  flow  to- 
gether easily.  Words  are  chosen 
for  their  precise  meaning. 


grammar  and  Word 
Choice 


For  the  most  part,  explanations 
of  cona‘pts  and  theories  arc 
accurate  and  complete.  Some 
helpful  applications  are  included. 


No  significant  errors  are  made 
Listeners  recognize  any  errors  to 
be  the  result  of  nervousness  or 
oversight. 


Explanations  of  concepts  and^oj 
theories  are  inaccurate  or  incom 
plete.  Little  attempt  is  made 
tic  theory  to  practice.  Listencr| 
gain  little  from  the  presentation. 


Enough  errors  are  made  dis; 
tract  a knowledgeable  listener^ 
but  some  information  is  accurate 
The  presentation  is  useful  if  ihe^ 
listener  can  determ inc  what  t 
information  is  reliable, 


For  the  most  part,  sentences  are 
complete  and  grammatical,  and 
they  flow  together  easily.  With  a 
few  exceptions,  words  are  chosen 
for  Iheir  precise  meaning. 


Listeners  can  follow  the  presei| 
talion»  but  they  are  distracted  bj 
some  grammatical  errors  and 
of  slang.  Some  sentences  are  inC 
complete/ halting,  and/orvoc|b 
uiary  is  somewhat  limited  or  - 
inappropriate. 
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l#jL/Kt  fa- 7 Lontnmea 


n- 

Levels  of  Achievement 

Criteria 

3 

Sophisticated 

2 

Competent 

1 

Not  Yet  Competent 

^reedom  from  Bias 
e.g.,  Sexism,  Racism, 
i\gism,  Hctcrosexism,  etc.) 

Both  oral  language  and  body 
language  are  free  from  bias. 

Oral  language  and  body  language 
are  free  from  bias  with  one  or 
two  minor  exceptions. 

Oral  language  and  / or  body 
language  includes  some 
identifiable  bias.  Some 
listeners  will  be  offended- 

®eriDH(?/  Ay^mance 

Personal  appearance  is  completely 
appropriate  for  the  occasion  and 
the  audience. 

Personal  appearance  is  generally 
appropriate  for  the  occasion  and 
audience.  However,  some  aspects 
of  appearance  reflect  a lack  of 
sensitivity  to  nuances  of  the 
occasion  or  expectations  of  tire 
audience. 

Personid  appearance  is  inapprb 
priate  for  the  occasion  and 
audience. 

RfspoHsiwHt^ss  to  Atttiifuce 

Verbal 

Interaction 

CcHisistently  clarifies,  restates, 
and  responds  to  questions. 
Summarizes  tchen  needed. 

Generally  responsive  to  audicircc 
comments,  questions,  and  needs. 
Misses  some  opportunities  for 
interaction. 

Responds  to  questions 
inadequately. 

Body  Language 

Bod}'’  language  reflects  comfort 
interacting  with  audience. 

Body  language  reflects  some  dis- 
comfort interacting  with  audience. 

Body  language  reveals  a rclu(>' 
tance  to  interact  with  audience. 

flA'V'l  rs  J^y|  0-1  ■ /\^  I T anrTtlruKirt  I 
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b-2  i^uwic  tor  tngine  uesjgn  frpjea 


■ 

Levels  of  Achievement 

Criteria 

Excellent  (A) 
4 points 

Good  (B) 
3 points 

Needs  Improvement  (C,  D) 
2 points 

Unacceptable  (F) 
1 point 

Forff[!(/iihoif  of 
Design  Prol’lc'in 

Formulation  and 
>cope  of  problem 

Design  problem  form- 
ulation is  clear  and 
well  thought  out.  The 
problem  scope  is  well 
defined. 

The  problem  formulation 
is  dear,  but  the  scope  is 
not  well  defined. 

The  problem  formu  la  tion 
is  unclear  in  some  respects 
imd  does  not  appear  to  be 
well  though  tout. 

The  design  problem  is 
not  formulated  dearly. 

significance: 

The  problem  chosen 
represents  a curi'ftrt 
challenge  facing  the 
engine  industr)'.  The 
potential  market  is  large 
and  dearly  identified. 

The  problem  represents  a 
airrent  challenge  in  the 
engine  industiv',  but  the 
potcntLal  market  is  small 
or  is  not  dearly  identified. 

Die  problem  does  not 
represent  a current 
challenge  in  the  engine 
industry,  and  the  market 
is  small  or  is  not  clearly 
identified. 

Die  problem  does  not 
represent  a current 
challenge  in  the  engine 
industry.  There  is  no 
explanation  about  who 
would  be  interested  in 
the  product  or  why  they 
should  buy  it.  There  is  no 
evidence  of ! he  back- 

ground  work  (e.g.^  market 
analysis)  that  is  needed 
to  design  an  engine. 


Contimd 


Jule  Scarborough,  2006 


nCUKE  6-2  Contimml 


Levels  of  Adiievemenl 

Criteria 

Excellent  (A) 
4 points 

Good  (B) 
3 points 

Needs  Improvement  (C,  D) 
2 points 

Unacceptable  (F) 
1 point 

n^immn^  Ski!! 
Jtilization 

t 

Vnnlysis 

Engineering  analysis  is 
detailed  and  challenging 
and  IS  used  at  every  stage 
of  the  design  process. 

The  engmccring  analysis 
is  detailed  and  challenging, 
but  some  steps  do  not 
appear  to  be  supported 
by  calculations 

Some  analysis  is  mcluded, 
but  it  is  not  very  detailed 
orchallengmg  Many 
steps  arc  not  supported 
by  calculations. 

Engineering  analysis  % 
infrequently  used.  When 
used,  it  appears  trivia! 
and  leads  to  obvious 
conclusions. 

Documentation 

Documentation  is 
thorough  and 
complete. 

There  is  some  missing  in- 
formation in  the  docu- 
mentation. 

There  is  a great  deal  of 
missmg  information  in  the 
documentation 

Documentation  is  pooi  eft 
nonexistent. 

Assumptions 

All  assumptions  are  stated 
and  justified. 

Assumptions  are  stated, 
but  some  are  not  justified. 

Assumptions  itre  stated, 
but  none  are  justified. 

No  assumptions  are 
stated. 

Extension  of  Knowledge 
ibontlnlcrnal 
EombusHon  Engine^i 

i 

Concepts  beyond  those  in 
the  prerequisite  course 
arc  frequently  used.  The 
professor  may  have 

Prerequisite  coiuse  content 
IS  used  easily,  and  some 
material  beyond  the 
course  is  included. 

Prerequisite  course  content 
is  used,  but  new  and 
unfamilier  areas  are  not 
introduced. 

Prerequisite  course 
content  is  not  applied 
correctly  New  areas 
not  included. 

learned  sometinns  new. 


Jule  Scarborough,  2006 


!'mn  hlalls 


Group  functioning 

The  group  functions  well. 
Peer  review  indicates  good 
distribution  of  effort.  All 
membei  s are  challenged 
and  feel  their  contribu- 
tions are  valued. 

Regularity  and 
>roductivity  of 
neetings 

The  group  meets  regularly 
and  the  meetings  are 
productive. 

Jse  of  group 

iroblcm-soiving 

echniques 

The  group  makes  frequent 
use  of  brainstorming  and 
group  problem-solving 
techniques  and  documents 
the  effect  of  these  sessions. 

Vritteii  Cowwivication 

)rganization 

Written  work  is  well 
organized  and  easy  to 
understand. 

definition  of  terms 

All  new  terms  arc  defined. 

The  group  functions  fairly 
well.  Some  people  in  the 
group  believe  they  are 
working  harder  (or  less; 
hard)  than  others,  but 
everyone  is  contributing. 


The  group  meets  regularly, 
but  meetings  are  not  as 
productive  as  they  could 
be.  Some  members  are  not 
prepared. 

The  grottp  uses  brain- 
storming and  group 
problem-solving  techni- 
ques but  does  not  always 
document  the  effect  of 
tlieSe  sessions. 


The  organization  is  gen- 
erally good,  but  some 
parU  seem  out  of  place. 

Some  terms  arc  used 
without  definition. 


The  group  is  still  function- 
ing, but  each  individual  is 
doing  his/her  own  work 
and  ignoring  the  efforts  of 
others.  There  are  frequent 
episodes  where  one 
person's  design  will  not  fit 
with  another's  due  to  lack 
of  communication. 

The  group  meets  irregu- 
larly. Meetings  arc  not  as 
productive  as  they  could 
be  because  se\’eral 
members  are  not  prepared 

Some  attempt  to  use  group 
problem-solving  lechiuques 
is  observed,  but  decisions 
arc  not  based  on  results  of 
problcm-^K'ing  sessions. 


The  organization  is 
unclear. 


Many  terms  are  used  but 
not  defined. 


The  group  functions 
poorly.  AJl  work  is  the 
product  of  individual 
efforts. 


lie  group  does  not 
regularly,  and  \\'hen  it 
does,  some  members  a^: 
absent  and  no  one  is 
prepared 

No  attempt  to  use  group; 
problem-solving  te^bl 
ques  is  made.  Meeting^' 
are  worthless. 


The  report  is  disorganized 
to  the  extent  that  it  pre^ 
vents  understanding  oh 

content. 

Terms  are  used  without 
definition  to  the  extent  * 
that  understanding  is 
inhibited, 

Confinnea 
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iontmued 


Criteria 


Integration  of 
writing  styles 


Grammar 


Use  of  appendices 


Levels  of  Achievement 


Excellent  (A)  Good(B)  Needs  Improvement  (C,  D)  Unacceptable  (F) 

4 points  3 points  2 points  1 point 


The  team  developed  a 
writing  style  that  IS  uni- 
form throughout  the 
report.  There  is  no  indica- 
tion that  the  report  in- 
volved multiple  authors, 

The  work  has  been 
thoroughly  spell-chcckcd 
and  proofread  by  every- 
one in  the  group. 


Infomidtion  is  appropri- 
ately placed  m either  the 
main  text  or  an  appendix. 
Appendices  are  docu- 
mented and  referenced 
m the  text. 


There  is  some  indication  of 
multiple  authors  (c  g., 
different  fonts,  different 
paper,  etc.). 


There  are  a few  speOing 
and  grammatical  errors. 


Informabon  is  appropri- 
ately placed  in  either  the 
main  text  or  an  appendix. 
Documentation  and  ret- 
erenemg  in  text  are  some 
whatil’icomplete'^ 


There  is  ample  indication 
of  multiple  authors  (e.g., 
different  lonts,  different 
paper,  etc.), 


There  is  more  than  one 
spelling  or  grammatical 
error  per  page 


There  is  some  misplace- 
ment of  information  in  the 
text  Vi  the  appendix. 
Appendices  are  poorly 
documented  ana  refer- 
efutedintext 


Report  is  clearly  the  \vork 
ofmulbple  authors  with 
different  writing  styles, 
margins,  printer  fonts, 
and  paper  types. 

There  arc  frquent  mis- 
spelled words  and  serious 
grammatical  errors, 
indicating  that  time  was 
not  taken  to  spell-check 
and  proofread. 

Considerable  amount  of 
material  is  misplaced 
Appendices  are  not 
dcKumented  or 
referenced  in  texL 
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Oral  Cofiiminication 


Interest/ 

Design  presentation  is 

organization 

clear,  interesting,  and 
well  organized.  It  .starts 
and  ends  well. 

Visual  aids 

Visual  aids  are  used 
frequently.  They  are  easy 
bread  and  understand, 
and  they  are  of  profes- 
sional quality. 

Length 

The  presentation  iswibin 
the  assigned  time  limits. 

Engineering 

Engineering  analysis  is 

analysis 

presented  with  sufficient 
detail  tobeunderstixid, 
but  not  so  that  it  insults 
the  audience. 

The  design  presentation 
is  interesting,  but  some 
points  are  unclear.  The 
introduction  and/or 
conclusion  are  weak. 

Visual  aids  are  good,  but  a 
few  are  sloppy  or  difficult 
to  read. 


The  presentation  is  too 
shorter  too  long  by  two 
minutes  or  more. 

Engineering  iinalysis  is 
poorly  explamed  or  so 
detailed  that  the  audience 


The  design  presentation 
has  some  interesting  points 
but  is  difficult  to  follow. 
Either  the  introduction  or 
conclusion  is  missing. 

Most  visual  aids  are 
sloppy  and  hard  bread. 


The  presentation  Is  loo 
short  or  too  long  by  five 
minutes  or  more. 

Engineering  analysis  con- 
sists of  trivial  calculations 
and  is  piwly  explained. 


Van  GVfpen.  1999) 


The  design  presentation 
is  hard  to  follow  and 
poorly  organized.  It 
appears  to  be  off-the-c 
There  is  no  introduction 
or  conclusion. 

There  are  too  few  visual 
aid.s,and  tho.se  used  are 
carelessly  prepared. 


The  presentation  is  too 
short  or  too  long  by  ten 
minutes  or  more. 

No  engineering  analysis 
is  presented. 
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CEET  Initiative  on  Teaching  and  Learning 
Test  Item  Analysis  - Eeb.  16,  2006 

A.  Some  Textbook  Discussion  of  Item  Analysis 


B.  Purposes  of  Item  Analysis 


C.  Factors  Affecting  Item  Statistics 


D.  Item  Analysis  Example 

Definition  of  Item  Difficulty 
Definition  of  Item  Discrimination 
Item  Discrimination  - A Closer  Look 


E.  Complete  Item  Analysis  of  AHP  Test 
Discussion  of  Plugged  Items 


P.  Analyze  Items  in  Your  Exams 


G.  Validity,  Reliability,  and  Standard  Error  of  Measurement 


H.  Using  Testing  Services’  Scanning  and  Analysis  Services 
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Definitions  of  Item  Difficulty  and  Item  Discrimination 


Item  Difficulty 

The  difficulty  index  of  an  item  is  the  proportion  of  all  points  awarded  from  the  item  to  the 
maximum  possible  points  that  could  be  awarded  from  the  item.  Ranges  from  0 to  1 ; expressed  as 
a proportion  or  a percentage. 

Some  suggest  it  should  be  called  the  item  easiness  index  instead  of  the  item  difficulty  index  (but 
this  never  really  caught  on).  Often  referred  to  as  the  item’s  p-value. 

You  may  have  seen  the  definition  as:  the  proportion  of  students  who  answered  the  item  correctly 
to  the  total  number  of  students.  But  this  is  only  valid  when  all  items  are  worth  exactly  one  point 
each.  This  is  referred  to  as  0-1  (zero-one)  scoring  - an  incorrect  response  is  worth  zero  points,  a 
correct  response  is  worth  one  point.  No  partial  credit. 


Item  Discrimination 


Generically,  it’s  the  relationship  (the  correlation)  between  the  students’  performance  on  the  item 
and  the  students’  proficiency  in  the  content  the  item  measures.  The  item  should  discriminate 
between  the  proficient  student  and  the  non-proficient  student:  proficient  students  should  do  well 
on  the  item,  non-proficient  students  should  not  do  well. 

But  how  do  we  know  if  a student  is  proficient?  Appears  to  be  a little  circular  reasoning  here 
because  we  administer  the  item  to  determine  if  the  student  is  proficient  in  the  content.  Well,  if 
we  had  some  external  indicator  of  student  proficiency,  we  could  use  that.  But,  traditionally,  we 
use  the  actual  students’  scores  on  the  entire  exam  as  our  measure  of  student  proficiency  in  the 
content. 

This  is  a long  way  of  getting  to  our  operational  definition  of  item  discrimination.  Item 
discrimination  is: 

The  correlation  between  students’  scores  on  an  item  and  students’  scores  on  the  entire  exam.  For 
correlation  we  use  the  classic  Pearson  product-moment  correlation  coefficient  - CORREL( ) in 
Excel. 
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Discrimination  Demo 
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1 

38 

0 

39 

1 

39 

1 

39 

1 

39 

1 

Disc.  = 

0.01 

Disc.  = 

-0.23 

Disc.  = 

0.40 

Disc.  = 

0.12 

Disc.  = 

0.06 

Disc.  = 

-0.43 

Disc.  = 

0.54 

Diff  = 

0.98 

Diff  = 

0.98 

DIff  = 

0.98 

DIff  = 

0.80 

Diff  = 

0.80 

□Iff  = 

0.80 

DIff  = 

0.80 
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Factors  Affecting  Item  Statistics 


Instruction 


Degree  of  Match  Between 
What  Was  Taught  and  What 
Was  Tested 


Quality  of  Test  Items 


Students 


Item  Statistics 


AHP  Exam  1 - Item  Analysis  Results  for  Class  Discussion 

The  following  items  were  interesting  for  one  reason  or  another.  Three  of  them,  as  indicated, 
were  not  included  in  the  computation  of  scores;  they  were  essentially  removed  from  the  exam 
making  the  total  possible  points  52  instead  of  55.  The  three  items  that  were  removed  had  item 
difficulty  values  of  40%  or  less.  Almost  all  of  the  other  items  (those  not  included  here)  had  item 
difficulty  values  of  70%  or  greater,  i.e.,  they  were  answered  correctly  by  70%  of  the  class. 


This  item  shouldn’t  have  been  difficult,  but  several  students  missed  it. 

Answered  incorrectly  by  six  students  (item  difficulty  = 54%): 


1.  Compute: 


7(2- 6^3) -^3"  -2(10- 2-3) -2-4 +10^(2  + 3)-l 

■ ■ J 


28  - 8 + 1 


21 


Answer:  21 


The  performance  of  this  item  tells  me  I didn’t  teach  the  concept  very  well. 

(Not  included  in  scores.)  Answered  incorrectly  by  nine  students  (item  difficulty  = 31%): 

3.  A person’s  age  is  an  example  of  which  type  of  measurement  scale?  (Circle  one.) 

Nominal  Ordinal  Interval  (fRatio^ 


Shouldn’t  have  been  difficult;  temperature  is  discussed  explicitly  on  page  8 in  the  textbook. 

Answered  incorrectly  by  five  students  (item  difficulty  = 62%): 

4.  A person’s  body  temperature  in  Celsius  is  an  example  of  which  type  of  measurement  scale? 
Nominal  Ordinal  (Mterv^l)  Ratio 


This  turned  out  to  be  a very  complicated  way  to  test  for  comprehension  of  the  concept  of  a 
biased  statistic.  It  didn’t  work! 

(Not  included  in  scores.)  Answered  incorrectly  by  nine  students  (item  difficulty  = 31%): 

10.  When  we  compute  each  standard  deviation,  5,  of  an  infinite  number  of  samples  of  size  n 
from  a population  with  mean  ji  and  standard  deviation  a we  divide  the  SS  by  (n  -1)  instead 
of  by  n.  But  when  we  compute  the  mean,  X , of  each  sample  we  divide  the  sum  of  scores 
by  n instead  of  by 

(n  -1).  By  using  these  divisors  we  ensure  that 

Tad  the  mean  of  all  sample  means  is  /r  and  the  mean  of  all  sample  standard  deviations  is  a. 

K the  mean  of  all  sample  means  is  /r  and  the  mean  of  all  sample  standard  deviations  is 
less  than  a. 

c.  the  mean  of  all  sample  means  is  //  and  the  mean  of  all  sample  standard  deviations  is 
greater  than  a. 
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d.  the  mean  of  all  sample  means  is  less  than  //  and  the  mean  of  all  sample  standard 
deviations  is  less  than  a. 

e.  the  mean  of  all  sample  means  is  greater  than  jj.  and  the  mean  of  all  sample  standard 
deviations  is  greater  than  a. 

Several  students  placed  the  z-score  to  the  left  of  0,  when  it  should  be  to  the  right.  Same  for  item 
23. 

Answered  incorrectly  (or  partially  incorrectly)  by  five  students  (item  difficulty  = 69%): 

22.  In  a standard  normal  distribution  what  is  the  probability  of  obtaining  a z score  less  than  .90? 

Prob  = .50  + area  to  .90 

= .50  + .3159  = .8159 


Complicated,  yes.  But  I believe  there  was  an  appropriate  amount  of  practice.  So  what  happened? 

Answered  incorrectly  (or  partially  incorrectly)  by  eleven  students  (item  difficulty  = 54%): 

24.  If,  in  a standard  normal  distribution,  the  probability  of  obtaining  a z score  between  .67  and 
another  z score  greater  than  .67  is  .20  (20%),  what  is  the  other  z sco^*^ 

Area  to  z = .67  is  .2486;  Area  between  .67  and  desired  z 
must  be  .20 
Tberefore: 

(area  to  desired  z)  - (area  to  .67)  = .20,  and 
(area  to  desired  z)  = (area  to  .67)  + .20 
= .2486  + .20  = .4486 


Using  area  in  table  of  .4484,  and  desired  z = 1.63. 


0 ,67 


Window  dressing  is  usually  inappropriate  but  I wanted  you  to  discriminate  among  the  data 
provided.  Most  didn’t! 

(Not  included  in  scores.)  Answered  incorrectly  by  twelve  students  (item  difficulty  = 8%): 

37.  In  the  regression  of  Y on  X in  a sample  of  1000  subjects  the  following  data  were  obtained: 
Sx  = 13.5,  rxy  = .70,  and  the  standard  deviation  of  the  conditional  distribution  of  Y scores  at 
each  X score  is  4.28.  What  is  the  value  of  the  standard  error  of  estimate? 


Answer:  4.28 


Tbe  standard  deviation  of  the  conditional  distribution  of  Y scores  at  each  X score  is,  by 
definition,  the  standard  error  of  estimate.  All  of  the  other  data  presented  is  just 
window  dressing. 
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This  is  a good  item  - requires  you  to  think,  as  do  many  of  the  items.  Maybe  some  thought  too 
much! 

Answered  incorrectly  by  seven  students  (item  difficulty  = 46%): 

38.  If,  in  a sample  of  400  subjects,  the  correlation  between  variables  T and  U is  (+0.62)  and  the 
correlation  between  variables  V and  U is  (-.75)  then 

a.  the  regression  of  U on  T will  be  more  accurate  with  smaller  errors  of  estimate  than  the 
regression  of  U on  V. 

the  regression  of  U on  T will  be  less  accurate  with  larger  errors  of  estimate  than  the 
regression  of  U on  V. 

c.  the  regression  of  U on  T will  be  more  accurate  with  larger  errors  of  estimate  than  the 

regression  of  U on  V. 

d.  the  regression  of  U on  T will  be  less  accurate  with  smaller  errors  of  estimate  than  the 

regression  of  U on  V. 
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Item  Analysis  Example 


1 . Enter  label  information 

Item  labels  & Points  Possible  for  each  item  - across  columns. 
Student  ID  codes  - down  rows. 

[Reformat  as  needed,  e.g.,  column  widths  - row  heights.] 

2.  Enter  raw  data  - points  obtained  by  each  student  for  each  item. 
(Erom  the  raw  data  sheet.) 

3.  Enter  marginal  information  to  the  right  - down  rows: 

Totals  Across  Items  (Student  Scores). 

Total  points  possible  (max  score  possible  on  the  test). 

Student  scores  (sum  of  the  item  scores  for  each  student). 
Percents. 


4.  Enter  marginal  information  across  the  bottom  - across  columns. 
Total  Across  Students  (Item  Scores). 

Total  Points  Possible  (max  score  possible  for  each  item). 
Item  Scores  (sum  of  the  student  scores  for  each  item). 
Percents,  or  proportions  (Item  Difficulty  indices). 

5.  Enter  overall  totals  in  lower  right  comer. 

Sum  of  student  scores. 

Sum  of  item  scores. 

Max  possible  test  points  (either  sum  of  max  item  scores  or 
# students  * max  test  score). 

Percents. 


6.  Enter  Item  Discrimination  indices  in  the  lower  margin  - across  columns. 
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Purposes  of  Item  Analysis 


Item  Analysis  can: 


1 .  Provide  information  (item  difficulty  and  discrimination  indices) 
about  the  characteristics  of  each  item  and  how  the  items  are 
functioning 


2.  Provide  a basis  for  an  informative  class  discussion  of  the  test 
results. 


3.  Identify  gaps  in  student  preparation  - areas  needing  possible 
remediation. 


4.  Identify  gaps  in  instruction  (or  even  the  broader  curriculum). 


5.  Identify  items  needing  revision  or  elimination. 


6.  Improve  the  instructor’s  test  construction  skills. 
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Reliability  and  the  Standard  Error  of  Measurement 


Reliability  basically  refers  to  the  consistency  with  which  a test  measures  whatever  it  measures. 
(Validity  addresses  whether  the  test  measures  what  it  is  intended  to  measure.) 


Three  basic  types  of  reliability: 


Stability  Reliability  - consistency  over  time.  Also  called  test-retest  reliability. 

Administer  the  test  twice  to  the  same  students.  Reliability  is 
obtained  by  correlating  the  scores  from  the  two  administrations. 

Alternate  Forms  Reliability  - consistency  over  test  forms.  When  you  have  two  forms 
of  the  same  test  administer  both  forms  to  the  same  students. 
Reliability  is  obtained  by  correlating  the  scores  from  the  two  forms 

Internal  Consistency  - the  extent  to  which  the  items  in  a test  are  internally  consistent, 
homogeneous,  measure  the  same  general  construct.  There  are 
various  formulas:  KR20,  KR21,  Alpha,...  . All  yield  similar 
results. 


The  Standard  Error  of  Measurement  is  based  on  the  reliability  index  and  the  variation  of  the 
total  test  scores  for  the  group  of  examinees  who  took  the  test.  The  formula  is: 

SEM  = (TotalTestVariance)^Jl  - reliability 

This  is  interpreted  as  the  amount  of  error  expected  in  the  score  from  a single  student. 
Statistically,  it  is  the  standard  deviation  of  the  scores  obtained  if  we  could  measure  a single 
student  over  and  over  and  over.  If  we  assume  that  this  large  (infinite)  number  of  scores  is 
normally  distributed,  then  we  can  develop  confidence  intervals  for  the  student’s  true  score: 

68%  Confidence  Interval  = [(Student’s  Obtained  Score)  +1SEM] 

95%  Confidence  Interval  = [(Student’s  Obtained  Score)  +2SEM] 

99%  Confidence  Interval  = [(Student’s  Obtained  Score)  +3SEM] 
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Test  with  only  one  item 

■ 

Test  with  only  two  items 

■ 

Item  Scores  (25  pts.) 

Total  Test  Scores 

■ 

Item  1 Scores  (20  pts.) 

Item  2 Scores  (20  pts.) 

Total  Test  Scores 

Students 

■ 

Students 

A 

19 

19 

A 

20 

18 

38 

B 

22 

22 

B 

18 

18 

36 

C 

18 

18 

C 

13 

15 

28 

D 

16 

16 

D 

13 

17 

30 

E 

25 

25 

E 

16 

18 

34 

F 

23 

23 

F 

18 

19 

37 

G 

14 

14 

G 

15 

15 

30 

H 

20 

20 

H 

16 

16 

32 

1 

21 

21 

1 

14 

16 

30 

J 

15 

15 

1 

J 

17 

16 

33 

K 

17 

17 

1 

K 

19 

18 

37 

L 

19 

19 

L 

16 

17 

33 

M 

24 

24 

1 

M 

14 

13 

27 

Item  Discrimination 

Item  Discrimination 

1.00 

0.94 

0.89 

= CORREL( ) 
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ITEM  and  TEST  DEVEOPMENT 


CEET  Faculty  Development  Program 
The  Scholarship  of  Teaching 
Student  Assessment  Program  Component 
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Suggestions  for  Constructing  Multiple  Choice  Items 


1.  The  stem  of  the  item  should  be  meaningful  by  itself  and  should  present  a definite 
problem. 

2.  The  item  stem  should  inelude  as  mueh  of  the  item  as  possible  and  should  be  free  of 
irrelevant  material. 

3.  Use  a negatively  stated  stem  only  when  signifieant  learning  outeomes  require  it. 

4.  All  the  alternatives  should  be  grammatieally  eonsistent  with  the  stem  of  the  item. 

5.  An  item  should  eontain  only  one  eorreet  or  elearly  best  answer. 

6.  Items  used  to  measure  understanding  should  eontain  some  novelty,  but  beware  of  too 
mueh. 

7.  All  distraeters  should  be  plausible.  The  purpose  of  a distraeter  is  to  distraet  the 
uninformed  from  the  eorreet  answer. 

8.  Verbal  assoeiations  between  the  stem  and  the  eorreet  answer  should  be  avoided. 

9.  The  relative  length  of  the  alternatives  should  not  provide  a elue  to  the  answer. 

10.  The  eorreet  answer  should  appear  in  eaeh  of  the  alternative  positions  an 
approximately  equal  number  of  times  but  in  random  order. 

11.  Use  sparingly  speeial  alternatives  sueh  as  “none  of  the  above”  or  “all  of  the  above.” 

12.  Do  not  use  multiple-ehoiee  items  when  other  item  types  are  more  appropriate. 


From 

Linn,  Robert  L.,  Miller,  M.  David  (2005).  Measurement  and  Assessment  in  Teaching  (9'*’ 
ed.).  New  Jersey:  Pearson  Prentiee  Hall 
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Test  Development 


Housekeeping  and  Test  Analysis  Posttest 
Broad,  Four-Day  Schedule: 

Today,  3/30/06:  Introduction  to  Objective  Items  & Item  Writing 

4/6/06:  Item  Writing 

4/20/06:  Item  Writing  (And  Test  Development?) 

4/27/06  Test  Development  - Midterm  and  Final 


Today’s  Schedule: 

Why  Use  Many  Discrete,  Objectively-Scored  Items? 

Taxonomies  Revisited 

Classify  Test  Items  According  to  Bloom’s  Taxonomy 
One  of  My  Statistics  Tests  (It’s  good,  but  not  perfect) 

What  is  A Valid  Test  Item 

Suggestions  for  Constructing  Multiple  Choice  Items 
Write  MC  Items 

Suggestions  for  Constructing  Short  Answer,  True-False  and  Multiple  True-False  It 

Write  Short  Answer,  TF,  and  MTF  Items 

Write  Items  - Any  Type  Discussed  Today,  Or  Variations 

Print  Out  Items  for  Me  (or  Send  to  jsgilmer@niu.edu) 
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Why  use  many  discrete,  objectively -scored  items  instead  of  fewer  essay 

or  performance -type  items? 


Advantages  of  Objective  Items: 

1.  Better  Content  Coverage  (Improves  Test  Validity  and  Reliabi 

2.  Scoring  is  Objective,  not  Subjective  (Improves  Test  Reliabili 

3.  Scoring  is  Efficient  and  Easy,  Often  Can  Be  Machine  Scored 


Disadvantages  of  Objective  Items: 

1.  Require  Time  to  Develop 

2.  Hard  to  Assess  Higher  Order  Thinking  or  Performance 

3.  Guessing 
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Teaching  Models 


Teaching  Styles 
Learning  Styles 


TESA 


Table  1.5  A Summary  of  The  Seven  Principles  of  Good  Practice 


R»rinc:«F>le 

Sehavidral  indices 

GloocJ  encourages 

s'CucJen't — 'TecuH'V’  contact: 
Frecfuent  contact  lin  and  . 
out  ot  classes 

F^emembering  students*  names 

Involving  students  in  lab  andl  tield  research  proj- 
ects 

Talcing  students  to  conventions,  regional  conter— 

ences 

<3; <=>od  (practice  encO'Urages 
coor>eration  among  stu— 

' dentsr  <3:olIat>orative. 
noncc3n~ip>etitive  lean~iing 
in  small  groups  and 
student— to-student  met— 
worfcs 

Disclosing  personal  va.lues,  >/vhen  appropriate- 
A.ttending  student— sponsored  events 
r^/lentoring  and  Sntormal  advising 

Encx>uraging  selt— disclosure  to  one  another 
facilitating  the  tormation  ot  study  groups 
.^Vssigning  group  projects 

Idsing  peer  eval-uation  technigues  -when  grading 
Teaching  through  group  discussion 
Promoting  student— to-^student  tutoring/lea.ching 
C3rading  by  criteria  and  not  by  interstudent 
comparison 

GSood  practice-  e-ncourages 
active  learning:  ~Teacl~iing 
mettt,ods  tt>at  reguire 
more  ttian  passive  listen- 
ing and  note  talcing  trom 
students 

fTeguiring  class  presentations 

.Assigning  papers  and  projects  that  promote 
critical  thinicing 

y^slcing  students  to  integrate  contemporary 

events  with  course  material,  discussing  real— 
lite  cases,  -etc. 

Assigning  term  projects  and  independent  stud- 
ies 

Good  practice  gives 

prompt  teedbacic:  >Vs— 
sessment  ot  baseline 
Icno-wledge.  treguent  test- 
ing ot  progress  in  lea.m— 
ing.  and  global  assess- 
ment ot  educational 
outcomes 

C3ood  practice  empbasizes 
time  on  tasic:  ^Setting  -ap— 

^ propriate  time  demands 
and  helping  students 
leam>  to  manage  their 
time 

Involving  students  in  research 

Giving  guizzes  and  homeworic  assignments 
ITetuming  examinations  and  papers  -vs^ithin  a 
weeic 

Providing  teedbacic  to  students  early  in  the 
term 

Writing  comments  on  exams  and  papers 
Pretesting  students 

dialling  or  e-mailing  students  v\/'ho  rniss  classes 
Establishing  deadlines  tor  completing  assign- 
ments 

d>lscussing  course  dema.nds  >vi-th  students 
1— ielping  students  set  challenging  goals 
Encouraging  practice  n.jns  betore  oral  reports 
Stressing  selt— regulation,  studying,  and  atten- 
dance 

-Good  practice  communi- 
cates high  expectations: 
Setting  reasonable  but 
high  standards  tor 
achievement 

Fwleeting  with  students  ^A^ho  tall  behind 
Warning  students  about  time  comm-itment  to 
the  course 

Stressing  high  standards  ot  achievement 
Establishing  performance  expectations  orally 
and  in  >vriting 

■—Ielping  students  set  challenging  goals 
Explaining  penalties  tor  missed  or  late  >worfc 
>^ssigning  -writing 

dialling  attention  in  class  to  excellence  by  class 
members 

C3ood  practice  respects  di— - 
'verse  talents  and  way^s 
ot  learning:  Providing  a 
variety  ot  learning  experi- 
ences and  . a.ssessment 
options 

Encouraging  questions 

Discouraging  otf— tasfc.  divisive  comments 

t— isingi  a variety  ot  teaching  method's 

Discussing  the  contributions  ot  vx^omen  ^and  mi- 
nority psycholog  ists 

Developing  and  using  alternative  teaching 
methods 

Exploring  students*  bacl<grounds,  learning 

styles,  and  outloofcs  — 

Chickering  & Gamson  (1987) 


I_  fanning) ^ 


’^Research 


Carnegie 
Institute 
On  Teaching 


Literature 

Study 


Che  :king-Re-planniijg 


Develop 
Faculty 
Development 
Program 


Program  Quality  Feedback  Loop 


earner  Analysis 


Orientation 

Him 


Faculty 

Self 

Assessment 


Student 

Assessment 

of 

Instruction 


Course 

Evaluations 


Course 

Grades 


Test 

Grades 


Assess 

Needs 


student  Learning 
Style  Analysis 


Multiculturalism 


Learner  Motivation 


f—T 


iDevelop  Student  Assessments 


Write  Learning 
Objectives 


ill 


► 


Develop 

Tests 


Syllabus  Analysis 


Teaching  Analysis 
2/20-2/24 


Bloom’s  Analysis 
Dale’s  Analysis 


Assessment  Analysis 


Knowledge  Analysis 


Faculty  Instructional 
Analysis 


Develop 

Performance  Tasks 
& Rubrics 

+ Other 
Assessments 


Grading 


''  Research 
& 

Evaluation 


Design 

Methodology 

5/25 


Develop 

Instructional 

Strategies 

T 

I 


- 


Conduct 
Experiments 
Eall  2006 
Collect  Data 


The  Sct]olarship  of  Teaphing 

Reflective  Practice  \ 

I CEETs  I 

Faculty  'pevelopment  Program 
‘Intentional  ^nd  Reversed”  Instructional 

Design  j 

(Scarborough,  2006)  i 
‘Backward”  Design  Process  (Dick  & Carey,  1996:  Wiggins  & 
McTighe,  2005,\1995; Deming  Quality  Cycke,  1950  s) 


Analyze, 

Interpret 

Data 

Report 

Results 


T 


L ^lannini 


> 

Revise  Knowledge 

T 

1 

Revise 

Revise 

Content 

1 

Assessments 

Instijuction 

Revise  Research 
QuestioJis/ 
Research  Design 


Ongoing  Change  Loop 


/'TScting'S^ 
N^alyzi 


jrjg/ 


Identify 
Changes 
Needed  to 
Eaculty 
Development 
Program 


Identify 
Changes 
Needed  to 
Instruction 


I 


x?Scling'S^ 

^ApeHyzirig^ 


''achieved 


Jule  Scarborough,  2006 


(Jensen,  1998,  Tomlinson,  1999) 


The  brain  seeks  meaningful  patterns  and  resists 
meaninglessness 

It  retains  isolated  or  disparate  bits  of  info,  but  it 
is  much  more  efficient  at  retaining  information 
that  is  “chunked” 

Responds  much  more  effectively  and  efficiently 
to  something  that  carries  deep  and  personal 
meaning,  something  that  is  life  shaping, 
relevant,  important,  or  taps  into  emotions 


Students  need  to  be  optimally  challenged 
with  activities  that  ask  them  to  risk  a leap 
into  the  unknown,  but  they  know  enough 
to  get  started  and  have  additional  support 
for  reaching  a new  level  of  understanding 


© 

© 

Increase  intrinsic 
motivation 

Increase  apathy  and 
resentm  ent 

CHOICES 

VS 

REQUIRED 

Provide  choices:  content, 
tim ing,  work  partners, 
projects,  environment,  or 
resources 

Directed  1 00%,  no 
student  input,  resources 
restricted 

RELEVANT 

VS 

IRRELEVANT 

Make  it  personal:  relate 
to  family,  neighborhood, 
city,  life  stages,  love, 
health,  etc 

Impersonal,  useless,  out 
of  context,  and  only  done 
to  pass  a test 

ENGAGING 

VS 

PASSIVE 

Make  it  emotional,  Disconnected  from  the 

energetic;  make  it  real  world,  low  interaction, 

physical;  use  learner-  lecture,  seatwork,  or  video 

imposed  deadlineSj^^___^_^^ 


Jule  Scarborough,  2006;  some  content  based  upon  J. Parker,  2001 


Qualities  of  Genius  (Armstrong,  1998) 


Curiosity 

Playfulness 

Imagination 

Creativity 

Wonder 

Wisdom 


• Inventiveness 

• Vitality 

• Sensitivity 

• Flexibility 

• Humor 

• Joy 


How  Genius  Shuts  Down 


Role  of  the  Home 

- Emotional  dysfunction 

- Poverty 

- Fast  track  lifestyies 

- Rigid  ideoiogies 


How  Genius  Shuts  Down 


Role  of  the  School 

-Testing  and  grading 
- Labeling  and  tracking 
-Textbooks  and  worksheet  learning 
-Tedium 


How  Genius  Shuts  Down 


Role  of  the  Popular  Media 

- Stereotypical  images 

- Insipid  language 

- Mediocre  content 


Awakening  Genius  in  the 

Classroom 

Re-awaken  genius  in  yourself 
Create  a genial  classroom  climate 

- Freedom  to  choose 

- Open-ended  exploration 

- Freedom  from  judgment 

- Honoring  every  student’s  experience 

- Belief  in  every  child’s  genius 

Genius  is  expressed  in  different  ways 


How  do  students  express  their 

genius? 


Multiple  IntelligeaQj^ 


• Much  research  has  been  done  and  drawn 
several  conclusions: 

-We  think,  learn,  and  create  in  different  ways 

- Development  of  our  potential  is  affected  by 
the  match  between  what  we  learn  and  how 
we  learn  with  our  particular  intelligences 

- Intelligence  is  multifaceted,  not  singular 

- Intelligence  is  fluid  and  not  fixed 


Ability  to  think  in  words  and  to  use  language  to 
express  and  appreciate  complex  meaning 

Words,  wordsmiths,  speaking,  writing,  listening, 
reading,  papers,  essays,  poems,  plays, 
narratives,  lyrics,  spelling  grammar,  foreign 
language,  memos,  bulletins,  newsletters, 
newspapers,  FAXes,  E-mail,dialogues,  debates. 

Authors,  poets,  journalists,  speakers, 
newscasters 


Makes  it  possible  to  calculate,  quantify,  consider 
propositions,  and  hypotheses,  and  carry  out 
complex  mathematical  operations 

Reasoning,  deductive  and  inductive  logic,  facts, 
data,  information,  spreadsheets,  databases, 
sequencing,  ranking,  organizing,  analyzing, 
proofs,  conclusions,  judging,  evaluations,  and 
assessments 

Scientists,  accountants,  engineers, 
programmers 


Instills  the  capacity  to  think  in  three  dimensional 
ways.  Enables  one  to  perceive  external  and 
internal  images,  to  produce  or  decode  graphic 
information 

Images,  graphics,  drawings,  sketches,  maps, 
charts,  doodles,  pictures,  designs,  imagination, 
visualization,  dreams,  films,  cartoons 


Sailors,  pilots,  sculptors,  painters,  architects 


Enables  one  to  manipulate  objects  and  fine  tune 
motor  skills.  Ability  to  unite  body  and  mind. 
Foundation  of  human  knowing  as  we  experience 
life  through  our  sensory-motor  experiences 

Experiential,  hands-on,  actions,  play,  touch, 
manipulate,  games,  field  trips,  drama,  sports 


Dancers,  athletes,  surgeons,  physical  educators 


Sensitivity  to  pitch,  melody,  rhythm,  and 
tone. 

Music,  rhythm,  pacing,  tenor,  choir,  songs, 
jingles. 

Composers,  sensitive  listeners, 
conductors,  musicians 


Capacity  to  understand  and  interact  effectiveiy 
with  others.  Operates  primarily  through  person- 
to-person  relationships  and  communication 

Interact,  laugh,  whisper,  empathize,  sympathize, 
group  projects,  debates,  dialogues 


Teachers,  social  workers,  actors,  counselors 


Ability  to  construct  an  accurate  perception  of 
oneself  and  to  use  such  knowledge  in  planning 
and  directing  one’s  life 

Self-reflection,  logs,  journals,  poetry, 
meditations,  creative  expression 

Psychologists,  theologians,  philosophers, 
parents 


Recognition,  appreciation,  and  understanding  of 
patterns  in  nature.  Understanding  natural  and 
human-made  systems 

Field  trips,  nature  walks,  ecological  studies, 
plant  identification,  weather  forecasting 

Botanists,  farmers,  zoologists,  landscapers 


Content  of  a Healthy 
Classroom 

relevant  to  students,  personal  and 
seems  familiar,  connected  to  the 
world  they  know 

helps  students  understand 
themselves  and  their  lives  more  fully 
now,  and  will  continue  to  do  so  as 
they  grow 


Content  of  a Healthy  Classroom 


authentic,  offering  “reai”  history  etc  not  just 
exercise  about  the  subject 

can  be  used  immediateiy  for  something 
that  matters  to  the  students 

makes  students  more  powerfui  in  the 
present  and  the  future 


Creating  a Healthy  Classroom 


• Appreciate  each  child  as  an 
individual 

• Teach  the  child  as  a whole 

• Continue  to  develop  expertise 

• Link  students  and  ideas 

• Strive  for  joyful  learning 


Creating  a Healthy  Classroom 


Help  students  make  sense  of  their  own 
ideas 

Share  teaching  with  students 
Strive  for  student  independence 
Use  positive  energy  and  humor 
Discipline  is  more  covert  than  overt 


Learning  Styles  (David  A.  Kolb,  1999) 


Two  ways  to  take  in  experiences 

- Concrete  Experience  (Feeling) 
-Abstract  Conceptualization  (Thinking) 

Two  ways  to  deal  with  experiences 

-Active  Experimentation  (Doing) 

- Reflective  Observation  (Reflecting) 


The  Lewinian  Experiential  Learning  Model 


Concrete 

experience 


Testing  implications 
of  concepts  in  new 
situations 


Observations  and 
reflections 


Formation  of 
abstract  concepts 
and  generalisations 


Kolb  (1984)  p.21 


Kolb’s  Model  of  Experiential  Learning 


Concrete 


Kolb  (1984)  p.42 


Experiential  Learning  as  the  Process  That  Links 
Education  Work  and  Personal  Development 


Personal 

Development 


Kolb  (1984)  p.4 


Dewey  Model  of  Experiential  Learning 


Kolb  (1984)  p.23 


Experience  (Feeling) 


Learning  by  experiencing 

- Learning  from  specific  experiences 

- Relating  to  people 

- Being  sensitive  to  feelings  and  people 

Learning  Situations 

- New  experiences,  games,  role  plays 

- Peer  feedback  and  discussion 

- Personalized  counseling 


Abstract  Conceptualization/Generalizing 

(Thinking) 


Learning  by  Thinking 

- Logically  analyzing  ideas 

- Planning  systematically 

-Acting  on  intellectual  understanding 

Learning  Situations 

- Theory  readings 
-Study  time  alone 

- Clear,  well-structured  presentation  of  ideas 


Active  Experimentation/Applying 

(Doing) 


• Learning  by  doing 

- Showing  ability  to  get  things  done 

- Taking  risks 

- Influencing  people  and  events  through  action 

• Learning  Situations 

- Opportunities  to  practice  and  receive  feedback 

- Small  group  discussions 

- Self-paced  learning  activities 


Reflective  Observation  (Reflecting) 


Learning  by  reflecting 

- Carefully  observing  before  making  judgments 

- Viewing  issues  from  different  perspectives 

- Looking  for  the  meaning  of  things 

Learning  Situations 

- Lectures 

- Opportunities  to  take  observer  role 

- Objective  tests 


Learning  Style  Quadrants 


How  to  Address  Learning  Styles 


To  ensure  that  all  student  learning  styles 
are  addressed  in  a class,  include  the 
following  sections: 

Experiencing 

Refiecting 

Generalizing 

Applying 


Learning  Styles  - Web  Sites 


http://www.learningstyle.com 

http://www.ncsu.edu/felder-public/ILSdir/stvles.htm 

http://www.ncsu.edu/felder- 
public/ILSdir/Zvwno  Validation  Studv.pdf 

http://www.ncsu.edu/felder-public/Papers/LS-1988.pdf 

http://www.indiana.edu/~intell/map.shtml 

http://www.enqr.ncsu.edu/learningstvles/ilsweb.html 

http://www.indstate.edU/ctl/stvles/learnina.html#LSHE 


• Select  one  shape  that  is  most 
representative  of  you  - the  one  you  relate 
to  most 


n 


Additional  Factors 


There  are  other  factors  that  influence 
learning 

- Stress:  biodot 

- Sleep:  teenagers  need  between  9 and  1 0 
hours  of  sleep  a night 


Teaching  Styles 


Just  as  students  are  smart  in  different 
ways  and  have  different  learning  styles, 
we  also  have  different  teaching  styles 


Teaching  Models 


Joyce,  b.,  Weil,  M.  with  Calhoun,  E.  (2004)  Models  of  Teaching.  Seventh  Edition. 


Faculty  Development  Program  : Models  of  Teaching 


Why  Models? 


“Structured,  logically  consistent, 
cohesive... patterns  of  teaching”  Joyce  and 

Weil  (1972) 

Holistic  Approach  to  teaching 

Ties  together  theory,  planning,  classroom 
management,  teaching  and  learning,  and 
assessment 


Families  of  Models 


Information  Processing  Family 

- Learning  to  think  by  thinking 

- Modeis  that  increase  students’  abiiity  to 
master  and  organize  information,  buiid  and 
test  hypotheses  etc 

Personal  Family 

- Models  that  focus  on  personal  identity 

- Promote  self-awareness  and  self- 
understanding 


Families  of  Models 


Behavioral  Systems  Family 

- Models  that  take  advantage  of  our  ability  to 
modify  behavior  in  response  to  tasks  and 
feedback 

Social  Family 

- Focus  on  our  social  nature  and  how  social 
interaction  can  enhance  learning 


Direct  Instruction 


Teacher  directed  and  controlled 

Major  goal  is  to  maximize  student  learning 
time 

High  priority  on  the  assignment  and 
completion  of  academic  tasks 

Atmosphere  of  relatively  neutral  affect 


Direct  Instruction  cont. 


Orientation 

-Objective,  content  relationships,  procedures 

Presentation 

- Concept  explanation,  demonstration 

Structured  practice 

-Teacher  leads  step  by  step 

Guided  practice 
Independent  practice 


Simulations 


• Designed  to  closely  mimic  reality 

• Complexity  can  be  controlled 

• Students  become  involved  in  situations 
similar  to  real  life 

• Teacher  must  raise  students’ 
consciousness  about  underlying  concepts 
and  principles 

• Popular  simulations:  Monopoly,  SimCity 


orougli,200i?^wwieoi 


Simulations  cont. 


Orientation 

- Present  concept  and  topics,  explain 
simulation 

Participant  training 

- Set  up  scenario,  assign  roles 
Simulation  operations 

- Conduct  activity,  obtain  feedback,  clarify 
misconceptions,  continue 

Participant  debriefing 


Inductive  Thinking 


• Students  learn  information  and  concepts 
through  the  act  of  classifying 

• Students  gather  and  classify  information  to 
build  and  test  hypotheses 

• A generic  model  because  classification  is 
applicable  to  many  different  disciplines 


Inductive  Thinking  cent. 


• Concept  Formation 

- Enumeration  and  listing 
-Grouping 

- Labeling,  categorizing 

• Interpretation  of  Data 

- Identifying  critical  relationships 

- Exploring  relationships 

- Making  inferences 


Scarborough, 


Inductive  Thinking  cent. 


Application  of  Principles 

- Predicting  consequences,  explaining, 
hypothesizing 

- Explaining  and/or  supporting  hypotheses 

- Verifying  prediction 


Concept  Attainment 


Challenges  students  to  distinguish  a 
concept  by  comparing  and  contrasting 
positive  and  negative  examples 

Students  determine  the  attributes  of  a 
category  that  already  exists 


Concept  Attainment  cont. 


Presentation  of  data  and  identification  of 
concept 

Testing  attainment  of  concept 
Analysis  of  thinking  strategies 


Inquiry 


Helps  students  inquire  independently  but 
in  a disciplined  manner 

Specializes  in  causal  reasoning, 
sharpening  tools  of  scientific  inquiry 

Teaches  students  a process  for 
investigating  and  explaining  phenomena 

Based  on  a conception  of  scientific  inquiry, 
this  model  teaches  skills  and  language  of 
scholarly  inquiry 


Jule  Scarborough,  2006;  some  content  based  upon  J. Parker,  2001 


Inquiry  cont. 


Confrontation  with  the  problem 

-explain  inquiry  procedures,  present  discrepant 
event 

Data  Gathering-verification 

Data  Gathering-experimentation 
- isolate  variables,  hypothesize  and  test 
Formulating  an  explanation 
Analysis  of  inquiry  process 


Jule  Scarborough,  2006;  some  content  based  upon  J. Parker,  2001 


Advance  Organizer 


Helps  teachers  organize  and  convey  large 
amounts  of  information  as  meaningfully 
and  efficiently  as  possible 

Helps  students  become  active  learners 
when  they  receive  information  through 
lectures  and  written  assignments 

Primary  means  of  strengthening  cognitive 
structure  and  enhancing  retention  of  new 
information 

Jule  Scarborough,  2006;  some  content  based  upon  J. Parker,  2001 


Advance  Organizer  cent. 


Presentation  of  advance  organizer 

- identify  attributes,  give  examples,  provide 
context,  repeat,  prompt  awareness  of 
knowledge 

Presentation  of  Task  or  Material 

- present  material,  make  logical  order  explicit, 
link  to  organizer 

Strengthen  Cognitive  Organization 

- integrate,  elicit  critical  approach,  clarify,  apply 

Jule  Scarborough,  2006;  some  content  based  upon  J. Parker,  2001 


Memory/Mnemonics 


Designed  to  increase  the  capacity  to  store 
and  retrieve  information 

Helps  students  develop  strategies  for 
acquiring  and  remembering  information 

Systematic  procedures  for  enhancing 
memory 


Jule  Scarborough,  2006;  some  content  based  upon  J. Parker,  2001 


Memory/Mnemonics  cont. 

Attend  to  material 

- underline,  list,  reflect  etc 

Develop  Connections 

- make  material  familiar  using  key-words, 
substitute  words,  or  link-words  techniques 

Expand  Sensory  Images 

- Ridiculous  association  or  exaggeration 

Practice  Recall 

- Practice  recall  until  completely  learned 


Jule  Scarborough,  2006;  some  content  based  upon  J. Parker,  2001 


Synectics 


Creative  problem  solving  process  using 
irrational  analogies  to  help  develop 
creative,  metaphoric  and  critical  thinking 

Teaches  metaphoric  thinking 

Consciously  breaks  from  routine  thinking 
to  generate  new  ideas 

Syn  - bringing  together 

Ectics  - diverse  elements 


Jule  Scarborough,  2006;  some  content  based  upon  J. Parker,  2001 


Synectics  - Creating  something 

new 

Description  of  present  condition 
Direct  analogy 
Personal  analogy 
Compressed  conflict 
Direct  analogy 

Re-examination  of  original  task 


Jule  Scarborough, 2006;  some  content  based  upon  J. Parker,  2001 


Synectics  - Making  the  strange 

familiar 

Substantive  input 
Direct  analogy 
Personal  analogy 
Comparing  analogies 
Explaining  differences 
Exploration 
Generating  analogy 


Jule  Scarborough,  2006;  some  content  based  upon  J. Parker,  2001 


Cooperative  Learning 


A teaching  arrangement  of  small  groups  of 
students  working  together  to  achieve  a 
common  learning  goal 

Emphasizes  team  spirit  rather  than 
individual  competition 

Tasks  require  that  students  depend  on 
one  another 


Jule  Scarborough,  2006;  some  content  based  upon  J. Parker,  2001 


Jurisprudential 


Helps  students  learn  to  think 
systematically  about  contemporary  issues 

Formulates  issues  as  public  policy 
questions  to  analyze  alternative  positions 
about  them 

Provides  students  with  tools  for  analyzing 
and  debating  social  issues 


Jule  Scarborough,  2006;  some  content  based  upon  J. Parker,  2001 


Jurisprudential  cent. 


Orientation  to  the  case 

Identifying  the  issues 

Taking  positions 

Exploring  the  stance 

Refining  and  qualifying  positions 

Testing  factual  assumptions  behind 
positions 


Jule  Scarborough,  2006;  some  content  based  upon  J. Parker,  2001 


Role  Play 


Exploring  problems  through  action 

Students  explore  their  feelings,  attitudes, 
and  values 

Develops  problem  solving  skills 

Offers  opportunity  to  resolve  interpersonal 
and  social  dilemmas 


Jule  Scarborough,  2006;  some  content  based  upon  J. Parker,  2001 


Role  Play  cont. 

Warm  up  group 
Select  participants 
Set  the  stage 
Prepare  the  observers 
Enact 

Discuss  and  evaluate 
Reenact 

Discuss  and  evaluate 

Share  experiences  and  generalize 

Jule  Scarborough,  2006;  some  content  based  upon  J. Parker,  2001 


Non-Directive 

The  hard  part  of  figuring  out  how  to  teach  is 
learning  when  to  keep  your  mouth  closed, 

which  is  most  of  the  time 

Carl  Rogers,  about  1 960 


Jule  Scarborough,  2006;  some  content  based  upon  J. Parker,  2001 


Non-Directive 


Focuses  on  facilitating  learning 

Teacher-student  relationship  is  more  like 
counselor  or  learning  partner 

Helps  students  attain  greater  personal 
integration,  effectiveness,  and  realistic 
self-appraisal 

Nurtures  students  rather  than  controlling 
the  sequence  of  learning 


Jule  Scarborough,  2006;  some  content  based  upon  J. Parker,  2001 


Non-Directive  cont. 


• Defining  the  helping  situation 

- Teacher  encourages  free  expression 

• Exploring  the  probiem 

- Student  defines,  teacher  accepts  and  clarifies 

• Deveioping  insight 

- Student  discusses  problem 

• Planning  and  decision  making 

- student  plans  initial  decision  making,  teacher 
supports 

• Integration 


Jule  Scarborough,  2006;  some  content  based  upon  J. Parker,  2001 


TESA 


Teacher  Expectation  Student 

Achievement 


Jule  Scarborough,  2006;  some  content  based  upon  J.ParkelfSOO 


TESA  Interaction  Model 


Five  units 
3 strands 

-A:  Response  opportunities 

- B:  Feedback 

-C:  Personal  Regard 

Each  unit  contains  a strategy  from  each 
strand 


Equitable  Distribution  of 
Response  Opportunities  (1) 

• Low  achievers  are 
less  likely  to  be  called 
on  than  high 
achievers 

• Teachers  call  on  male  ^ 

students  more 

frequently  than 
female  students 


Jule  Scarborough,  2006;  some  content  based  upon  J. Parker,  2001 


Positive 

• When  the  teacher  provides  a 
response  opportunity  for  one 
of  the  five  students  identified 
as  iow  achievers  or  one  of  the 
five  high  achievers 


Negative 

• When  the  teacher 
unreasonably  prohibits  a 
target  student  from 
responding  or  performing 


Jule  Scarborough,  2006;  some  content  based  upon  J. Parker,  2001 


Affirmation  or  Correction 


• The  teacher  should 
acknowledge  correct 
responses,  or 
whatever  part  of  the 
response  is  correct, 
and  try  to  elicit 
additional  or  improved 
information 


Positive 

When  the  teacher  informs 
the  student  who  has 
responded  to  a question 
that  his/her  response  or 
work  is  or  is  not 
acceptable 


Negative 

When  the  teacher  does  not 
react  or  comment  after  a 
student  has  responded  to 
a question 


Proximity 

Where  the  student  is 
seated  in  the 
classroom:  the 
nearness  of  the 
teacher  to  students 

I 


Jule  Scarborough<  2006,  some  content  based  upon  J. Parker,  2002001 


Positive 

• When  the  teacher 
comes  within  arm’s 
reach  of  a target  ^ 

student,  whether  or 
not  the  student  is 
aware  of  his/her 
presence 


Negative 


When  the  teacher 
avoids  proximity  with 
a target  student 


Jule  Scarborough<  2006,  some  content  based  upon  J. Parker, 


Individual  Helping  (2) 


• To  provide 
academic 
assistance  to  one 
student  at  a time 

• Teachers  should  try 
to  provide  individual 
help  to  low 
achievers  as 
frequently  as  other 
students 


Positive 

• When  the  teacher 
gives  individual 
assistance  to  a 
student 


Negative 


• When  the  teacher 
ignores  the  students 
attempt  to  obtain 
teacher  help 


Jule  Scarborough<  2006,  some  content  based  upon  J. Parker, 


Praise 


• Verbal  and 
nonverbal  feedback 
of  a student’s 
performance 

• Teachers  are  less 
likely  to  praise 
perceived  low 
achievers  and  more 
likely  to  criticize 
them  for  incorrect 
public  responses 


Positive 


• When  the  teacher 
praises  the  student’s 
learning  performance 


Negative 


When  the  teacher 
criticizes  the 
student’s 
performance  in  a 
sarcastic  or 
demeaning  manor. 


Courtesy 

Respect  of  and  for 
another;  politeness 

Use  courteous 
words  as  frequently 
with  low  achievers 
as  with  other 
students  and  as 
frequently  with  all 
students  as  with 
adults 


Positive 

When  the  teacher  uses 
expressions  of  courtesy  in 
interaction  with  the  student 


Negative 

When  the  teacher  behaves 
toward  the  student  in  a 
disrespectfui  manner  that 
would  not  be  characteristic 
of  the  teacher’s  behavior 
towards  adults 


Latency  (3) 


• “Wait  time”:  the  time 
that  elapses  between 
asking  a question  and 
terminating  the 
response  opportunity 


Positive 


When  the  teacher  allows  the 
student  enough  time  to 
think  the  question  over 
before  the  teacher 
terminates  the  response 
opportunity  or  attempts  to 
assist  the  student 

Negative 

When  the  teacher  allows  the 
student  less  than  five 
seconds  to  respond 


Reasons  for  Praise 


According  to 
Brophy(1986), 
students  should  be 
praised  when  they: 

- have  made  genuine 
progress; 

- may  not  realize  or 
appreciate  their 
accomplishments; 

- respond  well  to 
praise 


Positive 

• When  the  teacher 
gives  a reason  for 
praising  a student’s 
learning  performance 


Negative  I 


• When  the  teacher  is 
sarcastic  or  gives 
insincere  praise 


Jule  Scarborough<  2006,  some  content  based  upon  J. Parker, 


Personal  Interest  Statements 

and  Compliments 


The  teacher  gives 
compliments  or  makes 
statements  relating  to  a 
student’s  personal 
interests  in  recognition 
of  students  behaviors 
that  are  extraneous  to 
the  instructional  tasks 


Positive 

When  the  teacher  asks 
questions,  compliments,  or 
makes  statements  relating 
to  the  student’s  personal 
interests  or  experiences 

Negative 

• When  the  teacher 
negatively  curtails  or 
belittles  the  student’s 
attempt  to  tell  about  a 
personal  interest  or 
activity 


Delving,  Rephrasing,  Giving 

Clues  (4) 

• To  help  all  students  to 
respond  to  questions 
by  providing  them 
additional  information 


Positive 

• When  the  teacher  provides 
any  additional  information 
verbally  or  nonverbally  to 
help  the  student  respond  to 
a question 

Negative 

• When  the  teacher  terminates  the 
response  opportunity  of  a student 
who  has  not  responded  or  whose 
answer  was  inadequate  without 
rephrasing  the  question,  providing 
additionai  information,  or  deiving  in 
some  way 


Listening 


• The  Rule  of  Two- 
Thirds  states  that,  in 
the  average 
classroom,  someone 
is  talking  2/3  of  the 
time.  Two-thirds  of 
that  time,  the  person 
talking  is  usually  the 
teacher. 


Jule  Scarborough<  2006,  some  content  based  upon  J. Parker, 


Positive 

• When  the  teacher 
maintains  eye  contact  with 
the  student  or  indicates  to 
the  student  that  the 
response  was  heard 


Negative 

• When  the  teacher  is 
inattentive  to  a student 
whose  verbai 
communication  has  been 
invited  or  permitted 


Jule  Scarborough<  2006,  some  content  based  upon  J.Parkt 


Touching 


Touching  is  a form  of 
communication  (for 
example,  a pat  on  the 
back  to  show 
approval  or 
congratulations) 


Jule  Scarborough<  2006,  some  content  based  upon  J.Parki 


Positive 

• When  the  teacher 
touches  the  student  in 
a friendly  manner 

Negative 

• When  the  teacher 
rejects  the  student’s 
attempt  to  touch  the 
teacher  or  uses  touch 
as  punishment 


Jule  Scarborough<  2006,  some  sSffSSm  based  upon  J. Parker, 


Higher-Level  Questioning  (5) 


To  ask  a question  that 
stimulates  a students 
cognitive  reasoning 
skills 

Higher-level 
questioning  strategies 
provide  opportunities 
for  all  students  to 
think 


Positive 

When  the  teacher  asks  the 
student  a question  that 
requires  him/her  to  do 
something  more  than  merely 
remember  the  answer  from 
reading,  previous  teacher 
instruction,  or  another 
source 


Negative 

When  the  teacher 
implies  or  states  that 
questions  are  either 
easy  or  difficult 


Accepting  Feelings 


• Receptive  responses 
by  a teacher  showing 
that  he/she 
recognizes  the 
feelings  underlying  a 
particular  behavior 
and  acknowledges 
them 


4 


I 


Positive 

• When  the  teacher 
recognizes  and 
accepts  a student’s 
feelings  in  a 
nonevaluative  manner 

Negative 

• When  the  teacher 

discourages  or 
disparages  a 
student’s  feelings  


Jule  Scarborough<  2006,  some  content  based  upon  J.Parki 


Desisting 


“a  teacher’s  doing 
something  to  stop  a 
misbehavior” : in 
other  words,  a 
disciplinary  action  by 
the  teacher 


Jule  Scarborough<  2006,  some  content  based  upon  J. Parker, 


Positive 

• When  the  teacher  asks  a 
student  to  desist  from  a 
behavior  in  a calm, 
courteous  manner  that  does 
not  put  the  student  down 
and  does  not  imply  that 
misbehavior  was  expected 
of  him/her 


Negative 

• When  the  teacher  insults  the 
student  or  vents  anger  and 
hostility  on  the  student  in 
dealing  with  misbehavior 


Teaching  Standards 

Teaching  standards  for  your  discipline  are  in 

your  packets 

Ask  yourself  the  following  questions: 

- Do  I meet  the  standards  of  good  teaching  in  my 
discipline? 

- What  are  my  strengths  and  which  areas  do  I need 
to  develop? 

Meet  as  a discipline  and  discuss  your 

answers 

- Collectively,  what  do  your  answers  mean  for  your 
students? 


Active  Learning 


Cooperation  in  the  Classroom 


David  W.  Johnson 
Roger  T.  Johnson 
Karl  A.  Smith 

(Interaction  Book  Company,  1998) 

This  powerpoint  presentation  was  taken  directly  from  the  text  by 
Johnson,  Johnson,  and  Smith,  1998 

Jule  Scarborough,  2006 

(for  PowerPoint  presentations,  contact  juiescarborough@niu.edu) 


Causes  of  Missed  Opportunities  to 
Capitalize  on  the  Power  of  Groups 


1 . Belief  that  isolated  work  is  the  natural  order  of 
the  world. 

2.  Resistance  to  taking  responsibility  for  others. 

3.  Confusion  about  what  makes  groups  work. 

4.  Fear  that  they  cannot  use  groups  effectively  to 
enhance  learning  and  improve  teaching. 

5.  Concern  about  time  and  effort  required  to 
change. 

Jule  Scarborough,  2006 


Definitions 


^ Cooperation:  We  Sink  or  Swim  Together 

ndividuals  work  together  to  achieve  shared  goals;  maximize  own  and  others’  learning) 


^ Competition:  I Swim,  You  Sink;  I Sink,  You  Swim 

ndividuals  work  against  each  other  to  achieve  a goal  only  one  or  a few  can  attain) 

^ Individualistie:  We  Are  Each  in  this  Alone 

ndividuals  work  by  themselves  to  achieve  goals  unrelated  to  others’  goals) 


Jule  Scarborough,  2006 
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1 Table  1.1:  Comparison  Of  Old  And  New  I 

1 Paradigms  Of  Teaching  | 

Factor 

♦ 

- Old  Paradigm  Of 
Teaching 

' New  Paradigm  Of 
Teaching 

Knowledge 

Tranfc-fmdd  From  Facully  To 
^>mdenLs 

Joincty  Ccm-ilnKUcfi  By  Students 
Ami  Foulfy 

Students 

Passive  Vc.'isel  To  Be  FilU:<l  By 
l-aculiy'ii  KuowleUv.^ 
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TTifiKfnrmer  of  Own  Knowledge 

Nature  Ol* 
l^omiDg 

E 
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inrlividual;  llequircs  Exlrin^i. 
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1 

Any  Expert  CanMeiidi 

1 Ttaching  Ls  Complex  And 
j^^£cndiTN_ConM[d^ 

Jule  Scarborough,  2006 


The  New  Paradigm  of  Teaching 


■ Principal  Activities... 

• Knowledge  is  constructed,  discovered,  transformed,  and 
extended  by  students 

• Students  actively  construct  their  own  knowledge 

• Learning  is  a social  enterprise  in  which  students  need  to 
interact  with  the  instructor  and  classmates 

• Faculty  effort  is  aimed  at  developing  students’  competencies 
and  talents 


Jule  Scarborough,  2006 


The  New  Paradigm  of  Teaehing 


Principal  activities  cont.. 

• Education  is  a personal  transaction  among  students  and  between 
the  faculty  and  students  as  they  work  together 

• All  of  the  above  best  take  place  within  a cooperative  context 

• Teaching  is  assumed  to  be  a complex  application  of  theory  and 
research  that  requires  considerable  instructor  training  and 
continuous  refinement  of  skills  and  proeedures 


Jule  Scarborough,  2006 


Implementing  The  New  Paradigm: 
Cooperative  Learning 


♦ Learning  Together  or  Alone 

■ Students’  learning  goals  may  be  structured  to 
promote  cooperative,  competitive,  or  individualistic 
efforts 


■ Instructional  activities  are  aimed  at  accomplishing 
goals  and  are  conducted  under  a goal  structure. 


Jule  Scarborough,  2006 


Implementing  The  New  Paradigm 
Cooperative  Learning 


♦ Learning  Goal  (Outcome  or  Objective) 

■ A desired  future  state  of  demonstrating 
competence  or  mastery  in  the  subject  area  being 
studied 


Jule  Scarborough,  2006 
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Implementing  The  New  Paradigm 
Cooperative  Learning 


♦ Goal  Structure 

■ Specifies  the  ways  in  which  students  will  interact 
with  each  other  and  the  instructor  during  the 
instructional  session 


Jule  Scarborough,  2006 
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Implementing  The  New  Paradigm 
Cooperative  Learning 


♦ Cooperative  Learning 

■ Instructional  use  of  small  groups  so  that  students 
work  together  to  maximize  their  own  and  each 
other’ s learning 


Jule  Scarborough,  2006 
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Implementing  The  New  Paradigm 
Cooperative  Learning 


♦ Competitive  Learning 

■ Students  work  against  each  other  to  achieve  an 
academic  goal 

♦ Individualistic  Learning 

■ Students  work  by  themselves  to  accomplish 
learning  goals  unrelated  to  those  other  students 


Jule  Scarborough,  2006 
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Implementing  The  New  Paradigm 
Cooperative  Learning 

♦ Why  Use  Cooperative  Learning? 

■ Greater  effort  to  achieve  by  students 

■ More  positive  relationships  among  students 

■ Greater  psychological  health 


Jule  Scarborough,  2006 
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Implementing  The  New  Paradigm 
Cooperative  Learning 

♦ Types  of  Cooperative  Learning  Groups 

■ Formal  cooperative  learning 

• Multiple  sessions 

■ Informal  cooperative  learning 

• Ad-hoc 

■ Cooperative  base  groups 

• Long-term  (lyr-t)  (in  my  case  - one  semester) 

Jule  Scarborough,  2006 
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Implementing  The  New  Paradigm 
Cooperative  Learning 


♦ Types  of  Cooperative  Learning  Groups 

■ For  each  group  there  are  cooperative  learning  scripts 

(once  conducted  several  times,  they  become  automatic) 

■ Scripts  are  procedures  for: 

• Conducting  generic,  repetitive  lessons-writing  reports  or 
giving  presentations,  typical  routines,  etc. 

• Managing  classroom  routines,  e.g.  checking  homework, 
test  process  or  procedures,  reviewing  test,  standard  protocols 


Jule  Scarborough,  2006 


• • 


Implementing  The  New  Paradigm 
Cooperative  Learning 


♦ Types  of  Cooperative  Learning  Groups 

■ When  you  use  these  groups  repeatedly,  you  will 
gain  a routine-level  of  experience. 


■ Expertise  is  reflected  in  your  proficiency, 
competence,  and  skill  in  doing  something 


Jule  Scarborough,  2006 
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Implementing  The  New  Paradigm: 
Cooperative  Learning 


■ Expertise  in  structuring  cooperative  efforts  is  reflected 
in  your  being  able  to: 

• Take  any  lesson  in  any  subject  area  with  any  age  student  and 
structure  it  cooperatively 

• Use  cooperative  learning  60  to  80  percent  of  the  time 

• Describe  precisely  what  you  are  doing  and  why 

• Apply  the  principles  of  cooperation  to  other  settings,  such  as 
professional  relationships  and  staff  meetings. 


Jule  Scarborough,  2006 


Implementing  The  New  Paradigm 
Cooperative  Learning 

■ Expertise  can  be  achieved  by: 

• Teaching  a cooperative  lesson 

• Assessing  how  well  it  went 

• Reflecting  on  how  cooperation  could  have  been  better  structured 

• Teaching  an  improved  cooperative  lesson 


• Reassessing  lesson,  repeat  process 


Jule  Scarborough,  2006 
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Implementing  The  New  Paradigm 
Cooperative  Learning 

♦ What  Kind  Of  Group  Am  I Using? 

■ Pseudo-Learnin  g Group 

■ Traditional  Classroom  Learning  Group 

■ Cooperative  Learning  Group 

■ High  Performance  Cooperative  Learning  Group 

Jule  Scarborough,  2006 
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Implementing  The  New  Paradigm 
Cooperative  Learning 


♦ Pseudo-Learning  Group 

■ The  sum  of  the  whole  is  less  than  the  potential  of 
the  individual  members. 

■ Students  believe  that  they  are  rivals  with  each 
other  and  must  be  defeated. 


Jule  Scarborough,  2006 
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Implementing  The  New  Paradigm: 
Cooperative  Learning 


♦ Traditional  Classroom  Learning  Group 

■ The  sum  of  the  whole  is  more  than  the  potential  of 
some  members. 

■ Students  put  in  teams  where  little  or  ^ “real” 
teamwork  is  required  and  believe  they  will  be 
evaluated  as  individuals. 


Jule  Scarborough,  2006 


Implementing  The  New  Paradigm 
Cooperative  Learning 

Cooperative  Learning  Group 

■ More  than  the  sum  of  their  parts; 


■ All  students  perform  higher  academically  than 
alone. 

■ Students  believe  and  understand  that  their  success 
depends  on  the  efforts  of  all  group  members 

Jule  Scarborough,  2006 
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Implementing  The  New  Paradigm 
Cooperative  Learning 


High-Performance  Cooperative  Learning  Group 

■ Cooperative  learning  group  that  outperforms  all 
reasonable  expectations,  given  its  membership 


Jule  Scarborough,  2006 
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Implementing  The  New  Paradigm: 
Cooperative  Learning 


What  makes  Cooperation  Work 

■ Take  your  existing  lessons,  curricula,  and  courses  and 
structure  them  cooperatively 

■ Tailor  cooperative  learning  lessons  to  your  unique 
instructional  needs,  circumstances,  curricula,  subject 
areas,  and  students 

■ Diagnose  the  problems  some  students  may  have  in 
working  together  and  intervene  to  increase  the 

effectiveness  of  the  student  learning  groups  Jule  Scarborough,  2006 


Implementing  The  New  Paradigm 
Cooperative  Learning 


♦ What  makes  Cooperation  Work 

■ 5 essential  lesson  elements: 

• Positive  interdependence 

• Individual  and  group  accountability 

• Promoting  interaction  between  members 

• Teaching  interpersonal  and  small  group  skills 

• Group  Processing 


Jule  Scarborough,  2006 
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Implementing  The  New  Paradigm 
Cooperative  Learning 


♦ Positive  Interdependence 

■ Group  members  perceive  they  are  linked  with 
each  other  in  which  everyone  succeeds  together 
or  not  at  all 


Jule  Scarborough,  2006 
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Implementing  The  New  Paradigm 
Cooperative  Learning 


♦ Individual  and  Group  Accountability 

■ The  performance  of  each  individual  student  is 
assessed  and  used  to  ascertain  who  in  the  group 
needs  more  assistance,  support,  and 
encouragement. 

■ Students  learn  together  so  that  they  can 
subsequently  perform  higher  as  individuals. 


Jule  Scarborough,  2006 
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Implementing  The  New  Paradigm 
Cooperative  Learning 


♦ Promotive  Interaction 


■ When  members  share  resources  and  help 
support,  encourage,  and  praise  each  other’ s 
efforts  to  learn 


Jule  Scarborough,  2006 
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Implementing  The  New  Paradigm: 
Cooperative  Learning 


♦ Teaching  Interpersonal  and  Small  Group  Skills 

■ In  cooperative  learning  groups  students  are  required 
to  learn  academic  subject  matter  (taskwork)  and  also 
to  learn  the  interpersonal  and  small  group  skills 
required  to  function  as  part  of  a group  (teamwork). 


Jule  Scarborough,  2006 


Implementing  The  New  Paradigm 
Cooperative  Learning 


♦ Group  Processing 

■ Group  members  discuss  how  well  they  are 
achieving  their  goals  and  maintaining  effective 
working  relationships. 


Jule  Scarborough,  2006 
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Understanding  Cooperative  Learning 
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Implementing  The  New  Paradigm 
Cooperative  Learning 

Managing  Conflicts  Constructively 

■ Students  must  be  taught  the  procedures  and  skills  to: 

• Manage  the  academic/intellectual  conflicts  inherent  in 
learning  groups 


• Negotiate  constructive  resolutions  to  their  conflicts  and 
mediate  classmates’  conflicts 

♦ Train  students  to  negotiate 

♦ Train  students  to  mediate 


Jule  Scarborough,  2006 
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Table  1.2  Conflict  Resolution  Procedures 


Controversy 

Peacemaker  Pragra  : ^| 

Problem-Solving . 
Negotiations 

; Peer  Mediation 

Research  And  Prepare 
Positions 

State  What  You  Want 

End  Hostilities 

Present  And  Advocate 
Position 

State  How  You  Feel 

Ensure  Commitment  To 
Mediation  Process 

Open  Discussion: 
Advocate,  Refute,  Rebut 

State  Your  Reasons  For 
Wanting  And  Feeling 
As  You  Do 

Facilitate  Problem- 
Solving  Negotiations 

Reverse  Perspectives: 
Present  Opposing 
Position 

Reverse  Perspectives: 
Summarize  Opposing 
Position 

Formalize  Contract 

Reach  Consensus  On 
Best  Reasoned 
Judgment:  Synthesize 

Create  Three  Optional 
Agreements  That  Maxi- 
mize Joint  Outcomes 

i 

Choose  One  And 
Formalize  Agreement 

Jule  Scarborough,  2006 


Implementing  The  New  Paradigm: 
Cooperative  Learning 


♦ The  Cooperative  College: 

■ Uses  cooperative  learning  the  majority  of  the  time  in 
the  classroom 

■ Forms  teaching  teams,  task  forces,  and  ad  hoc 
decision-making  groups  within  a college 


■ Implements  administrative  cooperative  teams  within 
the  college 

Jule  Scarborough,  2006 


Gaining  Expertise  in  Cooperative  Learning 


1 Year 

Traininij  And  AiliiiHcatiori  ^ 

One 

instructors  (and  administrators)  are  organized  into 
teaching  teams  that  meet  weekly  and  are  trained  in  the 
fundamentals  of  cooperative  learning.  The  instructors  become 
an  inhouse  demonstration  project  for  other  instructors  to  view 
and  then  emulate. 

Two 

Insti'uctors  are  trained  in  the  integrated  use  of  (a)  formaJ, 
informal,  and  base  groups,  (b)  cooperative,  competitive,  and 
individualistic  learning,  as  well  as  the  teaching  of  advanced 
social  skills.  Administrators  receive  six  da>'s  of  training  on  the 
cooperative  college. 

Three 

Instructors  are  trained  in  how  to  (a)  create  academic  conflicts 
to  intellectually  challenge  students,  and  (b)  use  a peer 
mediation  program  to  manage  classroom  and  college 
discipline  problems.  The  "superstar"  instructors  receive 
leadership  training  on  how  to  (a)  conduct  the  training  on 
cooperative  learning,  (b]  give  inclassroom  help  and  support  to 
the  instructors  being  trained,  and  (c)  organize 
teaching  teams.  The  leaders  are  then  given  responsibility  for 
conducting  the  training  in  their  district. 

Four 

- 

Continued  functioning  of  colleagial  support  groups  supported 
by  the  instructors  trained  to  be  leaders. 

Jule  Scarborough,  2006 


Active  Learning 


Formal  Cooperative  Learning  Chapter  2 


David  W.  J ohnson 
Roger  T.  J ohnson 
Karl  A.  Smith 


(Interaction  Book  Company,  1998) 

This  powerpoint  presentation  was  taken  directly  from  the  text  by  J ohnson,  J ohnson,  and  Smith,  1998 
Jule  Scarborough,  2006 

(for  PowerPoint  presentations,  contact  juiescarborough@niu.edu) 


Formal  Cooperative  Learning  Groups 


■ I nstructor's  Role:  Being  "A  Guide  On  The  Side 

■ Make  preinstructional  decisions 

■ Explain  the  task  and  cooperative  structure 

■ Monitor  and  intervene 


Evaluate  and  process 


Jule  Scarborough,  2006 


Formal  Cooperative  Learning 

Groups 

■ Preinstrucional  Decisions 

Choose  academic  objectives  and  social  skills  objectives: 

Choose  social  skills  by: 

■ Monitoring  the  learning  groups 

■ Diagnosing  the  skills  needed  to  solve  problems  students  are  having  in 
working  with  each  other 

■ Asking  students  to  identify  social  skills  that  would  improve  their 
teamwork. 

■ Keeping  a list  of  social  skills  you  teach  to  every  class. 

■ Analyzing  what  social  skills  are  required  to  complete  the  assignment 


Jule  Scarborough,  2006 


Formal  Cooperative  Learning 

Groups 

■ Deciding  on  the  size  of  the  group 

■ The  smaller  the  better: 

■ With  the  addition  of  each  member,  the  resources  to 
help  the  group  succeed  increase 

■ The  shorter  the  period  of  time  available,  the  smaller  the 
learning  group  should  be 

■ The  smaller  the  group,  the  more  difficult  it  is  for 
students  to  hide 


Jule  Scarborough,  2006 


Formal  Cooperative  Learning 

Groups 


The  larger  the  group,  the  rrore  skillful  group  members 
must  be 

The  larger  the  group,  the  less  the  interaction  among 
members 

The  materials  available  or  the  specific  nature  of  the  task 
may  dictate  a group  size 

The  smaller  the  group,  the  easier  it  is  to  identify  any 
difficulties  students  have  in  working  together 


Jule  Scarborough,  2006 


The  Group  size  Wheel 


Jule  Scarborough,  2006 


Formal  Cooperative  Learning 

Groups 

■ Assigning  students  to  groups 

■ Advantages  to  heterogeneous  groups 

■ Students  are  exposed  to  a variety  of  ideas,  multiple 
perspectives,  and  different  problem  solving  methods 

■ Generate  more  cognitive  disequilibrium 

■ Engage  in  more  elaborative  thinking,  give  and  receive 
more  explanations,  and  engage  in  more  frequent 
perspective- taki  ng 


Jule  Scarborough,  2006 


Formal  Cooperative  Learning 

Groups 

■ Methods  of  assigning  students  to  groups 

■ Random  assignment 

■ Stratified  random  assignment- (note  on  tech  KSAs) 

■ Preferences 

■ I nstructor  selected  groups 

■ Self-selected  groups 

■ Length  of  life  groups 

■ Combination 


Jule  Scarborough,  2006 


Formal  Cooperative  Learning 

Groups 


■ Assigning  roles  to  ensure  interdependence 


■ Do  not  assign  roles  to  let  students  get  used  to  working 
together 

■ Assign  only  simple  roles  to  students 


■ Add  to  the  rotation  a new  role  that  is  slightly  more 
sophisticated,  e.g.  "checker  for  understanding" 


■ Add  formulating  and  fermenting  roles  that  do  not  occur 
naturally  in  the  group, e.g.  "relater  of  K"  Juie Scarborough, 2006 


Formal  Cooperative  Learning 

Groups 

Solving- preventing  problems  in  working  together 

■ Reduce  problems  such  as  one  or  more  members' 
making  no  contribution  to  the  group  or  one  member 
dominating  the  group 


■ Ensure  that  vital  group  skills  are  enacted  in  the 
group  and  that  group  members  learn  targeted  skills 


Create  role  interdependence  among  group  members 
by  assigning  members  interconnected  rolesuie Scarborough, 2oo« 


Formal  Cooperative  Learning 

Groups 

■ Arranging  the  room  by: 


■ Using  labels  and  signs  to  designate  areas 

■ Using  colors  to  attract  visual  attention 

■ Taping  lines  on  the  floor 

■ Using  mobiles  and  forms 

■ Using  lighting 

■ Moving  furniture 


Jule  Scarborough,  2006 


Formal  Cooperative  Learning 

Groups 

■ Planning  the  instructional  materials 


■ You  create: 

■ Materials  interdependence 

■ I nformation  interdependence 

■ I nterdependence  from  outside  enemies 
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Explaining  The  Task:  Flowchart 


Explain  the  assignment. 
The  assignment  needs  to  be 
a clear,  measurable  task. 


j 4.  Explain  the  procedures 
I students  are  to  follow  in 
completing  the  assignment. 


\ 


5.  Require  a visible  product 
that  each  student  signs. 
This  keeps  students  on  task 
and  helps  ensure  they  will 
behave 


ppn 


2.  Explain  lesson  objectives  to 
ensure  transfer  and 
retention.  Objectives  may 
be  stated  as  outcomes— "Ai 
the  end  of  this  lesson  you 
will  be  able  to  explain  the 
causes  of  the  French  and 
Indian  War. " 


II  Hit 


V 


Explain  the  concepts, 
principles,  and  strategies 
students  need  to  use  during 
the  lesson  and  relate  them 
to  students'  past  experience 
and  leamii 


6.  Ask  class  members  specific 
questions  to  check  their 
understanding  of  the 
assignment. 

✓ 
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7.  Ask  students  to  answer  in  pairs  or  triads  the  questions 
the  lesson  will  focus  on  to  (a)  establish  expectations  about  | 
what  the  lesson  will  cover  and  (b)  organize  in  advance 
^^vha^m^^nowabout^m 


structuring  Task  & 
Cooperative  Structure 


■ Explaining  Criteria  for  Success 


■ Criterion- referenced  or  categorical  judgments 

■ Letter  grades 

■ Group  improvement 

■ Reaching  a metric  or  standard 
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structuring  Task  & 
Cooperative  Structure 


■ Structuring  Positive  I nterdependence 


■ Positive  goal  interdependence 

■ structure  "We" 

■ Supplement  with  other  interdependent  elements,  e.g. 
"reward" 

■ Peer  encouragement  results 
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structuring  Task  & 
Cooperative  Structure 


■ Structuring  I ndividual  Accountability 


■ Make  each  group  member  a stronger  individual 
his/her  own  right  by: 

■ Assessing  performance  of  each  member 

■ Giving  the  results  back 

■ Comparing  to  criteria 

■ Providing  direct  Feedback 
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structuring  Task  & 
Cooperative  Structure 


■ Structuring  I ntergroup  Cooperation 


■ Encourage  members  to  find  groups  : 

■ Who  are  not  finished  and  help  them  understand  how  to 
complete  the  assignment  successfully 


■ Who  are  finished  and  compare  answers  and  strategies 
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structuring  Task  & 
Cooperative  Structure 


■ Specifying  Desired  Behaviors 


■ When  you  use  cooperative  learning  you  must  teach  students 

the  small  group  and  interpersonal  skills  they  need  to  work 
effectively  with  each  other 


■ You  define  the  needed  teamwork  skills  operationally  by 
specifying  the  behaviors  that  are  appropriate  and  desirable 
within  learning  groups 


Be  Specific,  Start  Small,  Emphasize  Over  iearning^'''®®“*°'™®^'^“® 


The  Cooperative 


■ Cooperative  Procedures 


Lesson 


■ Checking  homework 

■ Engaging  in  discussions 

■ Taking  notes 

■ Reading  assigned  material 

■ Drilling  and  reviewing 

■ Writing  compositions 

■ Resolving  intellectual  conflicts 

■ Executing  projects 


Jule  Scarborough,  2006 


The  Cooperative 


Lesson 


■ Checking  Homework 


■ Task:  Students  bring  completed  home  work  and 
understand  how  to  do  it  correctly 


■ Cooperative:  students  meet  in  base  group  to 
ensure  everyone  understands  how  to  complete  the 
assignment  correctly 
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The  Cooperative  Lesson 


■ Checking  Homework  cont.. 

■ Procedure 

■ Students  meet  in  groups 

■ One  member  acquires  materials  from  instructor 

■ Group  reviews  assignment  and  concentrates  on 
clarifying  parts  members  did  not  understand — 

using  roles  such  as  "explainer"  and  "accuracy  checker" 

■ Record  groups'  progress  on  assignment 

■ Returns  assignment  to  instructor 
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The  Cooperative  Lesson 


Checking  Homework  cont.. 

■ Expected  criteria  for  success: 


All  group  members  understand  how  to  complete 
each  part  of  the  assignment  correctly 


■ I ndividual  Accountability: 

Regular  examinations  and  daily  randomly 
selecting  group  members  to  explain  how  to 
solve  randomly  selected  problems 
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^ The  Cooperative  Lesson 

Turn  To  Your  Neighbor  Summaries 


■ Task:  Students  explain  their  answers  and 
reasoning  to  a classmate 


■ Cooperative:  create  a joint  answer  both  agree  on 
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The  Cooperative  Lesson 


■ Turn  To  You  Neighbor  Summaries  cont 

■ Procedure: 

■ Students  formulate  answer  to  a question 

■ Students  share  answer  with  a neighbor 

■ Listen  to  partner's  explanation 

■ Partners  create  new  answer  based  on  the  additional 
information 

Formulate.  Share.  Listen.  Create 
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^ The  Cooperative  Lesson 

Read  And  Explain  Pairs 

Task:  Learn  the  material  being  read  by  establishing 
the  meaning  of  each  paragraph  and  integrating  that 
meaning 


■ Cooperative:  Both  members  agree  and 
explain  meaning  of  material  jointly 
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The  Cooperative  Lesson 


■ Read  And  Explain  Pairs 

■ Procedure: 

■ Overview  material 

■ Student  A summarizes 

■ Student  B checks  accuracy 

■ Reverse  roles 

■ Summarize  and  agree  on  meaning  of  material 
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The  Cooperative 


Lesson 


■ Read  And  Explain  Pairs 


■ Expected  criterion  for  success:  Both 

members  are  able  to  explain  the  meaning 
of  the  material  correctly. 


■ I nstructors  should  systematically: 

■ Monitor  each  pair 


■ Ensure  individual  accountability 


■ Remind  students  about  intergroup  cooperation 
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The  Cooperative  Lesson 


■ Reading  Comprehension  Triads 


■ Task: 


■ Read  material 

■ Answer  questions 

■ Practice  checking  skills 
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The  Cooperative  Lesson 


■ Reading  Comprehension  Triads 

■ Cooperative: 

■ One  set  of  answers  from  group,  everyone 
agrees,  everyone  can  explain  each  answer 

■ +90%,  each  member  receives  bonus 

■ Each  member  assigned  a role 
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The  Cooperative  Lesson 

■ Reading  Comprehension  Triads 

■ Expected  criteria  for  success: 


Everyone  must  be  able  to  answer  each 
question  correctly 
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The  Cooperative  Lesson 


■ Reading  Comprehension  Triads 

■ I ndividual  Accountability 

■ One  member  randomly  chosen  to  explain 
group's  answer 

■ Individual  testing 

■ Each  member  required  to  explain  group's 
answers  to  another  group 
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The  Cooperative 


Lesson 


■ Reading  Comprehension  Triads 


■ Expected  behaviors:  Active  participation, 
checking,  encouraging,  and  elaboration  by 
all  members 


■ I ntergroup  cooperation:  Check 
procedures,  answers,  and  strategies  with 
another  group 
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The  Cooperative  Lesson 


■ Reading  Comprehension  Triads 

■ Also  known  as: 


■ Cooperative  Writing  and  Editing  Pairs 

■ Cooperative  Note-Taking  Pairs 
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The  Cooperative 


■ J igsaw  Procedure 


Lesson 


■ Task:  Students  to  learn  all  assigned 
material 


■ Cooperative  Goal : Each  member  to 
ensure  that  everyone  in  group  has  learned 
material 
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The  Cooperative 


■ J igsaw  Procedure 


Lesson 


■ Cooperative  Groups:  Distribute  unique  material  to 
each  member  in  group 


■ Preparation  Pairs:  Match  pairs  with  identical 
members  from  other  groups  (i.e.,  part  1 group  1 
with  part  1 group  2) 


■ Achieve  expertness  on  material 


■ Plan  how  to  teach  material  to  other  groups 
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The  Cooperative 


■ J igsaw  Procedure 


Lesson 


■ Practice  Pairs:  Students  pair  with  a new 
member  with  same  part,  practice  teaching 
material 


■ Cooperative  Groups:  Teach  expertise  to 
the  other  group  members  and  learn 
material  being  taught.  All  parts  of  the 
material  must  be  mastered 
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The  Cooperative 


■ J igsaw  Procedure 


Lesson 


■ Monitoring:  l nstructors  assist  students 
with  procedures 


■ Evaluation:  Assess  degree  of  mastery  of 
all  material  with  objective  test  taken 
individually.  +90%  receive  bonus 
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The  Cooperative 


■ Drill- Review  Pairs 


Lesson 


■ Task:  Correctly  solve  problems  or  engage 
in  procedures 

■ Cooperative:  Ensure  both  pair  members 
understand  the  strategies  and  procedures 
to  solve  the  problems  correctly 
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The  Cooperative 


■ Drill- Review  Pairs 


Lesson 


■ I ndividual  Accountability:  Instructor 
randomly  chooses  one  member  to  explain 
how  to  solve  a randomly  selected  problem 
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The  Cooperative  Lesson 


■ Drill- Review  Pairs 

■ Procedure 

■ Person  A explains  the  procedures  and 
strategies  required  to  solve  problem 

■ Person  B checks  accuracy  of  solution  and 
provides  encouragement  and  coaching 

■ A and  B reverse  roles 

■ Each  pair  compare  solutions  with  other  pairs 

■ Procedure  continues  until  all  problems  are 
completed 
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The  Cooperative  Lesson 


■ Academic  Controversies 

■ General  format  to  structure  Controversy 

■ Topic  with  pro  and  con  positions 

■ Groups  of  4,  pro/ con  topic  each  pair 

■ Assign  each  pair  tasks: 

■ Learn  its  position 

■ Research 

■ Prepare  argument 

■ Give  presentation 
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The  Cooperative  Lesson 

■ Joint  Project 

■ Task:  Complete  project 

■ Cooperative:  create  project  groups,  assign 
roles 

■ Criteria  for  success:  A completed  project 
that  each  group  member  can  explain 


Consider  Performance  Tasks  and  Rubrics 
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The  Cooperative  Lesson 


■ Joint  Project 

■ I ndividual  Accountability 

■ Each  group  member  responsible  for  proving 
involvement 

■ Each  member  presents  project  to  another 
member  or  performs 

■ Students  individually  tested  or  performance 
scored  using  rubric 
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The  Cooperative 


■ J oint  Skills 


Lesson 


■ Expected  Social  Skills:  Presenting  ideas, 
eliciting  ideas,  and  organizing  work 


■ I ntergroup  Cooperation:  Check 

procedures,  information,  and  progress  with 
other  groups 
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Monitoring  And  I ntervening 

■ Monitoring  Students'  Behavior 

■ Four  stages 

■ Preparing  for  Observing 

■ Observing 

■ Intervening  when  necessary 

■ Students  assess  quality  of  their  own 
participation 

■Use  Rubrics 
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Monitoring  And  I ntervening 


■ Guidelines 

■ Plan  observing  route  through  classroom 

■ Use  formal  observation  sheet  - rubric 

■ Do  not  obsess  over  behavior 

■ Detail  notes  with  specific  student  actions 

■ Train  and  use  student  observers  - rubric 

■ Allocate  time  to  discuss  with  observers 
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Monitoring  And  I ntervening 


■ Providing  Task  Assistance 


■ Cooperative  learning  groups  provide 
instructors  with  a "window"  into  students' 
minds 


I ntervening  to  Teach  Social  Skills 

■ Cooperative  learning  groups  provide 
instructors  with  a picture  of  students'  social 
skills 
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Evaluating  Learning  and 
Processing  I nteraction 


■ Provide  Closure  to  the  Lesson 

■ Assess  the  Quality  and  Quantity  of 
Learning 

■ Process  How  Well  the  Group  Functioned 

■ Feed  back 

■ Reflection 

■ I mprovement  Goals 

■ Celebration 
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Active  Learning 


Informal  Cooperative  Learning  Chapter  3 


David  W.  Johnson 
Roger  T.  Johnson 
Karl  A.  Smith 


(Interaction  Book  Company,  1998) 

This  powerpoint  presentation  was  taken  directly  from  the  text  by  J ohnson,  J ohnson,  and  Smith,  1998 


Jule  Scarborough,  2006  (for  PowerPoint  presentations,  contact  juiescarborough@niu.edu) 


The  Lure  of  Lecturing 

■ The  extended  presentation  in  which  the 
instructor  presents  factual  information  in 
an  organized  and  logically  sequential 
way 
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The  Lure  of  Lecturing 


■ Reasons  for  use: 

- Efficient 

- Flexible 

- Simple  to  implement 

- Makes  instructor  center  of  communication 
and  attention 
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Lure  of  Lecture 


■ Appropriate  use  of  Lecturing 

- Disseminate  information 

- Present  Material  that  is  not  available 
elsewhere 

- Expose  students  in  a brief  time  to  content 
integrated  from  a variety  of  sources 

- Expose  students  in  a brief  time  to  content 
too  complex  for  students  to  understand 
and  learn  on  their  own 
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Lure  of  Lecture 


- Demonstrate/model  strategies  and 
procedures  students  are  to  use  in  future 
assignments 

- Expose  students  in  a brief  time  to  several 
different  points  of  view 

- Arouse  students’  interest  in  the  subject 

-Teach  students  who  are  primarily  auditory 
learners 
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Lure  of  Lecture 


■ Parts  of  a Lecture 

- Introduction 

• Arouse  interest 

• Motivational  cues 

• Make  objectives  clear 

• Prompt  awareness  of  relative  knowledge 

• Use  advance  organizers 

- Concepts  given  to  the  student  prior  to  the  materiai  actually 
to  be  learned  that  provide  a stable  cognitive  structure  in 
which  the  new  knowledge  can  be  subsumed 
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Lure  or  Lecture 

■ Parts  of  a Lecture 


- Body 

• Cover  content  while  providing  a logical 
organization  for  the  material  being  presented 

- Conclude 

• Summarize  major  points,  recall  ideas,  give 
examples,  and  answer  questions 
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ITT! 


Lure  of  Lecturing 

■ Problems  With  Lecture 


- Attention  to  what  the  instructor  is  saying 
decreases  as  the  lecture  proceeds 

- Lecture  takes  an  educated,  intelligent 
person  orientated  toward  auditory  learning 
to  benefit 

-Tends  to  only  promote  lower  level  learning 
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Lure  of  Lecture 


■ Problems  With  Lecture  cont.. 

- Limited  by  assumption  that  all  students 
need  oral  information  at  the  same  time  and 
pace  without  dialogue 

- Students  tend  not  to  like  it 

- Based  on  assumptions  about  students 
cognitive  ability 
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Lure  of  Lecture 


Enemies  of  the  lecture 

- Preoccupation 

- Emotional  moods 

- Disinterest 

- Failure  to  understand 

- Feeling  of  isolation  and  alienation 

- Entertaining  lecture  that  misrepresent 
importance  of  material 
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Informal  Cooperative  Learning 

Groups 

■ Purpose 

-To  focus  student  attention  on  the  material 
to  be  learned 

- Set  a mood  conductive  to  learning 

- Help  organize  in  advance  the  material 
being  taught 

- Provide  closure  to  an  instructional  session 
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Figure  3. 1 Informal  Cooperative  Learning 


3:11 
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Focused  Discussion  2: 
Closure 


■ I 11 1 


Lecturing  With  Informal  Cooperative 

Learning  Groups 


■ Introductory  Focused  Discussion 

- Promote  advance  organization 

- Establishing  expectations  of  lecture 
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Lecturing  With  Informal  Cooperative 

Learning  Groups 

■ Intermittent  Focused  Discussions 

- Lecture  Segment  One 

• 10  to  15  minute  lecture 

- Pair  discussion  1 

• Each  student  formulates  answer 

• Students  share  answer  with  partner 

• Students  listen  to  partner’s  answer 

• Pairs  create  new  answer 
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Lecturing  With  Informal  Cooperative 

Learning  Groups 


■ Intermittent  Focused  Discussions 

- Repeat  steps  for  additional  segments  and 
pair  discussions  until  lecture  is  completed 


■Closure  Focused  Discussion 

-Students  complete  an  ending  discussion 
task 
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Step  One:  introductory  Focused 
Discussion 

■ Introductory  Focused  Discussion  Pairs 

-Task:  Answer  questions 

- Cooperative: 

• Formulate 

• Share 

• Listen 

• Create 
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Step  One: 

Introductory  Focused  Discussion 


Introductory  Focused  Discussion 
Pairs 

-Expected  Criteria  For  Success:  Each 
student  able  to  explain  answers 

-Individual  Accountability:  Random 
quizzing  of  individual  students 

-Expected  Behaviors:  Explaining, 
Listening,  synthesizing  by  all  members 

-Intergroup  Cooperation:  Compare  with 
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Step  One: 

Introductory  Focused  Discussion 

■ Question-And-Answer  Pairs 

-Task:  Answer  questions  on  homework 

- Cooperative: 

• Students  prepare  for  discussion 

• Students  randomly  assigned  to  pairs 

• Q&A  session  between  pairs 

• Instructor  provides  feedback 
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Step  One: 

Introductory  Focused  Discussion 

■ Question-And-Answer  Pairs 

- Expected  Criteria  For  Success:  Each 

student  writes  a paper  and  edits  group  members 
papers 

- Individuai  Accountability:  Each  student 

formulates  question  on  assignment,  partner 
answers  questions 

— Expected  Behaviors:  Exchange  questions, 
giving  good  explanations 
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Step  One: 

Introductory  Focused  Discussion 

■ Advanced  Preparation  Papers 

-Task:  Write  short  paper 

• Choose  topic  relating  to  assigned  reading 

- Major  theory 

- Concept 

- Research  study 

• Write  analysis  summarizing  material  and 
adding  material  from  another  source 
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Step  One: 

Introductory  Focused  Discussion 

■ Advanced  Preparation  Papers 

- Cooperative:  Students  will  check  each 
others  papers  in  learning  pairs,  checking 
for: 

• Paper  structure 

• Summary  of  theory 

• Clear  conceptual  definition  of  concepts  and 
terms 

• New  information  beyond  the  text 
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Step  One: 

Introductory  Focused  Discussion 


■ Advanced  Preparation  Papers 

• Expected  Criteria  for  success 

- Each  student  writes  a paper  and  edits  groupmates’ 
papers 

• Individual  Accountability 

- Each  student  writes  a paper 

- Each  student  edits  and  signs  another  paper 

- Have  students  explain  paper  to  another  group 
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Step  One: 

Introductory  Focused  Discussion 

■ Advanced  Preparation  Papers 

- Expected  behaviors:  critically  evaluating 
the  papers  of  groupmates 

- Intergroup  Cooperation:  Check  editing 
procedures  and  strategies  with  another 
group. 
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Progress  Checks 


■ ProgrGSS  Ch6Ck:  consists  of  questions  testing 
students’  knowledge  of  the  assigned  reading 

-Students: 

• Individually  complete  the  progress  check 

• Retake  the  progress  check  and  compare 
answers  with  group 

• Retake  with  the  whole  base  group  to  broaden 
discussion 
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Step  Two: 

Intermittent  Discussion  Pairs 

■ Intermittent  Discussion  Pairs 

- Students  form  pairs  and  are  given  a short 
discussion  task  to  be  completed  in  3 to  4 minutes 
to  ensure  that  students  actively  cognitively 
process  the  information  presented 

• Answer  a question  posed  by  the  instructor 

• Give  a reaction  to  the  theory,  concepts,  or 
information  being  presented 

• Elaborate  on  the  material  being  presented 
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Step  Two:  intermittent  Discussion  Pairs 


■ Using  intermittent  discussion  pairs  can 
solve  a number  of  problems  inherent  to 
lecturing... 


- Ensures  that  all  students  are  actively 
involved  in  learning  the  material  being 
presented  in  class 


Jule  Scarborough,  2006 


Step  Two:  intermittent  Discussion  Pairs 


■ Active  involvement  solves  three 
problems  with  class  discussions: 

- Lack  of  response  by  most  students 

- Domination  by  a few  students 

- Refusal  to  ask  question 
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Step  Two:  intermittent  Discussion  Pairs 


■ The  use  of  intermittent  discussion  pairs 
facilitates  the  understanding  and  retention  of 
material  being  learned 

■ Memory  interference 

- Retroactive  interference 

• Information  at  end  of  lecture  interferes  with 
information  from  beginning 

- Proactive  interference 

• Information  at  beginning  of  iecture  interferes 
with  information  from  end. 
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Step  Two:  intermittent  Discussion  Pairs 


The  use  of  intermittent  discussion  pairs  provides 
students  with  the  opportunity  to  receive  from 
classmates  frequent  and  immediate  feedback 

- Learning  tools 

• Turn  to  your  neighbor  summaries 

• Cooperative  note-taking  pairs 

• Read  and  explain  pairs 
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Step  Three:  Closure  Focused 

Discussions 

■ Closure  focused  discussions: 

-Assign  students  to  pairs  or  triads 

- Give  them  an  ending  discussion  task 
lasting  4 to  5 minutes 

• Learning  tools 

- Closure  note-taking  pairs 

- Closure  focused  discussion  pairs 

- Closure  cooperative  writing  pairs 

- Closure  review  pairs 

- Etc. 
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Other  Informal  Cooperative  Learning 
Groups 


■ Peer  Feedback  Groups 

- Students  tend  to  like  courses  that  offer 
frequent  opportunities  to  revise  and 
improve  their  work. 

- Student  learn  best  when  they: 

• Have  a chance  to  submit  earlier  version  of  their 
work 

• Get  detailed  feedback  and  criticism 
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Other  Informal  Cooperative  Learning 

Groups 


■ Peer  Feedback  Groups 

- Walberg  (1984)  identified  feedback  as  the 
most  powerful  predictor  of  learning 


- Students  need  continuous  feedback  about 
the  adequacy  of  their  performances  which 
may  be  best  provided  by  classmates 


Jule  Scarborough,  2006 


Other  Informal  Cooperative  Learning 

Groups 

Cooperative  Study  Groups 

-The  Harvard  Assessment  Seminars  (Light, 
1 990)  compared  the  grades  of  students 
who  studied  alone  with  those  of  students 
who  studied  in  groups  of  four  to  six. 

-Student  in  small  groups  performed 
better  than  students  who  worked  alone 

-Small  groups: 

• Spoke  more  often 

• Asked  more  questions 

• More  engaged  than  larger  groups 
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Cooperative  Base  Groups 


Chapter  4 

David  W.  J ohnson 
Roger  T.  J ohnson 
Karl  A.  Smith 


(Interaction  Book  Company,  1998) 

This  powerpoint  presentation  was  taken  directly  from  the  text  by  J ohnson,  J ohnson,  and  Smith,  1998 
Jule  Scarborough,  2006  PowerPoint  presentations,  contactjuiescarborough@niu.edu) 


Class  Base  Groups 


#The  larger  the  class  and  the  more  complex 
the  subject  matter,  the  more  important  it  is 
to  have  class  base  groups. 

#The  base  groups  meet  at  the  beginning  and 
ending  of  each  class  session  or  at  the 
beginning  of  the  first  class  session  each  week 
and  at  the  end  of  the  last  class  session  each 
week. 
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Class  Base  Groups 


#The  members  of  base  groups  should 
information  such  as  phone  numbers  and 
schedule  information  as  they  may  wish 
to  meet  outside  of  class. 
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Class  Base  Groups 


^All  members  are  expected  to  contribute 
actively  to  the  group's  work: 

■ Maintain  effective  working  relationships 
with  other  members 

■ Complete  all  assignments 

■ Assist  group  mates  in  completing  their 
assignments 

■ I ndicate  agreement  with  base  group's  work 
by  signing  weekly  contract. 
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Class  Base  Groups 


# At  the  beginning  of  each  session  students 
meet  in  base  groups  to: 

1.  Greet  each  other  - check  to  see  that  no  member 
is  under  undue  stress. 

2.  Complete  the  next  task  for  the  membership  grid. 
- hel  ps  members  get  to  know  each  other. 

3.  Pick  up  their  file  folders  with  an  attendance 
sheet,  feedback  form,  and  their  assignments 
from  the  previous  class  session. 
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Class  Base  Groups 


4.  Check  to  see  if  members  have  completed 
homework  or  need  help  doing  so 


5.  I n addition  to  homework,  members  review  what 
each  member  has  read  and  done  since  the  last 
class  session. 

Each  member  may  be  prepared  to: 

-give  a summary  of  what  he/ she  has  read, 
thought  about,  done 

-share  resources 

-share  copies  of  completed  assignments 
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Class  Base  Groups 


6.  Students  collect  each  members'  homework, 
record  it  in  their  Base  Group  Progress 
Report  Sheet,  and  place  the  assignments  in 
their  file  folder. 

The  file  folder  is  returned  to  the  instructors 
desk. 


Periodically  the  base  groups  may  be  given  a 
checklist  of  academic  skills  and  assesses  which 
ones  each  member  needs  to  practice. 
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Class  Base  Groups 


# Generally,  class  base  groups  are  available  to 
support  individual  group  members. 


■#  I f a group  member  arrives  late,  or  must  leave 
early  on  an  occasion,  the  group  can  provide 
information  about  what  that  student  missed. 


# Group  members  may  assist  one  another  in 
writing  require  papers  and  completing  other 
assignments. 
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Class  Base  Groups 


#The  class  session  closes  with  students 
nneeting  in  base  groups.  Closing  tasks  may 
be: 


■ Ensure  all  students  understand  the  assignments 

■ Summarize  at  least  three  things  members  learned 
in  today's  session. 

■ Summarize  how  members  will  use/'apply  what 
they  have  learned. 

■ Celebrate  the  hard  work  and  learning  of  group 
members. 
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Quick  Base  Group  Meetings 


^ At  times  there  may  be  only  a few 
minutes  for  base  groups  to  meet. 
Even  in  as  short  a time  as  five  to  ten 
minutes,  base  groups  are  given  four 
tasks. 

1.  A quick  self-disclosure  task  such  as: 

"What  is  the  most  exciting  thing  you  did 
during  your  vacation  break?" 
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Quick  Base  Group  Meetings 


2.  An  administrative  task  such  as  what  classes  to 
register  for  next  semester. 

3.  An  academic  task  such  as  , 'You  have  midterms 
coming  up.  As  a group,  write  out  three  pieces  of 
advice  for  taking  tests,  i wiii  type  up  the 
suggestions  from  each  group  and  hand  them  out 
next  week. " 

4.  A closing  task  such  as  wishing  each  other  good  luck 
for  the  day  or  week. 
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Building  A Group  Identity 


#The  effectiveness  of  base  groups  depends 
part  on  the  strength  of  the  group  identity. 


#The  first  week  the  base  groups  meet,  for 
example,  base  groups  can: 

■ Pick  a name 

■ Design  a flag 

■ Choose  a motto 
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Building  A Group  Identity 


# If  an  instructor  with  the  proper  expertise  is 
available,  the  groups  will  benefit  from 
participating  in  a "challenge  course"  involving 
ropes  and  obstacles. 


#This  type  of  physical  challenge  that  the 
groups  complete  together  builds  cohesion 
quickly. 
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Base  Group  Grid 


#The  more  personal  the  relationship  among 
base  group  members,  the  greater  the  social 
support  members  can  give  each  other. 

# While  students  will  get  to  know  each  other  on 
a personal  level  while  they  work  together,  the 
process  can  be  accelerated  through  the  use 
of  te  base  group  grid. 
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Base  Group  Grid 


Members 

Topicl 

Topic  2 

Topic  3 

Topic  4 

Frank 

Helen 

Roger 

David 
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Checking  and  Recording 
Homework 

1.  Homework  is  usually  checked  in  base 
groups  at  the  beginning  of  the  class  session 

2.  One  member  of  each  group,  the  runner, 
goes  to  the  instructor's  desk,  picks  up  the 
group's  folder,  and  hands  out  any  materials 
in  the  folder  to  the  appropriate  members. 
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Checking  and  Recording 
Homework 


1.  The  group  reviews  the  assignment  step-by-step 
to  determine  how  much  of  the  assignment  each 
member  completed  and  how  well  each  member 
understands  how  to  complete  the  material 
covered. 


Two  roles  are  utilized: 


• Explainer 

• Accuracy  checker 
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Checking  and  Recording 
Homework 

^Explainer  - explains  step-by-step  how 
to  complete  the  material  covered. 

^Accuracy  Checker  - verifies  that  the 
explanation  is  accurate  and  provides 
coaching  if  needed. 
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Checking  and  Recording 
Homework 


4.  At  the  end  of  the  review  the  runner  records  how 
much  of  the  assignment  each  member  completed, 
places  members'  homework  in  the  group's  folder, 
and  returns  the  folder  to  the  instructor's  desk. 
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Base  Group  Contract  Forms 


#At  the  end  of  each  class  session  the 
base  group  summarizes: 


■ What  they  learned 

■ How  they  will  apply  what  they  have 
learned 

■ How  they  will  help  each  other  implement 
what  was  learned 
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Base  Group  Folders 


# Provides  direct  communication  between 
students  and  instructor 


■#  Means  for  managing  attendance, 
assignments,  and  feedback. 


# I n each  folder  is  an  attendance  sheet  that 
each  member  initials  to  indicate  attendance 
at  the  session 
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Base  Group  Folders 


#At  the  end  of  each  session  students  place 
their  completed  feedback  form  in  the 
folder. 


■#The  feedback  form  may  ask  for 

■ Three  most  important  things  learned 

■ Favorite  part  of  the  session 

■ Questions  students  may  have 
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Value  of  Base  Groups 

1 

#There  are  many  reasons  why 
cooperative  base  groups  should  be 
used. 

■ I ncrease  student  achievement 

■ Build  more  positive  relationships  among 
students 

■ I ncrease  students'  psychological  health 
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Value  of  Base  Groups 


#Base  groups  may  also  be  used  to 


■ I ncrease  social  support 

■ Reduce  attrition 

■ Prorrote  positive  attitudes  toward 
education 
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Nature  of  Social  Support 


# Social  Support  - the  existence  and 

availability  of  people  whom  one  can  rely  for 
emotional,  instrumental,  informational,  and 
appraisal  aid. 


# Social  Support  System-  significant  others 
who  share  a person's  tasks  and  goals  and 
provide  resources  that  enhance  the 
individual's  well-being  and  help  the  individual 
mobilize  his/her  resources  to  deal  with 
challenging  and  stressful 

situations.  (J  ohnsonSdJ  ohnson,  1989)  Jule  Scarborough,  2006 


Nature  of  Social  Support 


^ There  are  two  types  of  social  support: 

1.  Academic  - classmates  and  faculty  provide 
the  assistance  and  help  students  need  to 
succeed  academically 

2.  Personal  - classmates  and  faculty  care 
about  and  are  personally  committed  to  the 
well-being  of  each  student. 
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Base  Groups  and  Social 
Support 


# The  more  social  support  a student  has,  the 
higher  the  student's  achievement  will  tend  to 
be,  the  more  the  student  will  persist  on 
challenging  tasks,  the  more  likely  students 
will  be  graduated , the  healthier 
psychologically  and  physically  the  students 
will  tend  to  be,  the  better  able  the  students 
will  be  to  manage  stress,  and  the  more  likely 
students  will  be  to  challenge  their 
competencies  to  grow  and  develop.  (Johnson 
$ Johnson, 1989) 
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Emotional  cone em  such  as 
attachment,  reassurance,  and  a 
sense  of  being  ahle  to  rely  on  and 
coitftde  in  a person,  all  of  which 
contribute  to  the  belief  that  one  is 

loved  and  valued. 

Zn.stim.men.ta.1  aid  such  as  direct 
aid,  goods,  or  services. 


Xnfor-matioxi.  aid  such  as  facts  or 
advice  that  may  help  to  solve  a 
problem. 


.Appraisal  aid  such  as  feedbaclc 
about  degree  to  w^hich  certain 
behavioral  standards  are  met 
(information  relevant  to  self- 
evaluation). 


G ROU  P'S 


Ns:  -Td  lisioi=?j^Asi?: 


Achi  evement  and  productivity, 

including  persistence  on  diflicult 
and  challenging  tasks. 


^Physical  health,  as  individuals 
involved  in  close  relationships  live 
longer,  get  sick  less  often,  and 
recover  from  illness  faster  than  do 

isolated  individuals. 

T^sychological  health , as  close 
relationships  promote  adjustment 
and  development  by  preventing 
neuroticism  and  psychopathology, 
reducing  distress,  and  providing 

resoTjjrces  such  as  confidants. 

Constructive  management  of 
stress  and  challenges  by 
providing  the  caring,  resources, 
infoi*mation,  and  feedback  needed 
to  cope  with  stress  and  by 
buffering  the  impact  of  stress  on 
the  individual.  Social  support  and 
stress  are  related  in  that  the 
greater  the  social  support 
individuals  have,  the  less  the 
strbss  thej?  experience,  and  the 
better  able  they  are  to  manage  the 
stresses  involved  in  life.  The  same 
is  true  of  challenges  that  test  the 
limits  of  a student’s  ability  and 
resolve.  There  are  few^  challenges 
that  cannot  be  met  when  sufficient 
social  support  is  provided. 
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Long-Term  Cooperative  Efforts 


#The  longer  a cooperative  group  exists, 
the  more  caring  their  relationships  will 
tend  to  be,  the  greater  the  social 
support  they  will  provide  for  each  other, 
the  more  committed  they  will  be  to 
teach  other's  success,  and  the  more 
influence  members  will  have  over  each 
other. 
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PERSONAL  Support 


ACADEMIC  Support 


Krico\>rHgf^  And  Hold  Member?  Personalize  Gass  And  \ Life 

Aijcniintable  To  Complete 
AsBigTunenU,  Attend  Clas5.  And 

iVchie\^»3  Acadernically 

Discusses  Assignments,  Ans>vers  Lij?ton  Sympruhohcally  WTien  A 
Questions,  Gi>e  Help  & Assistance  In  Member  Has  Problems  With  Friends 
Understanding  Material  Being  Or  Parent^' 


Communicate  Cemmitment  To  One's 

W*e])»J8eing 

Discuss  Personal  Beliefs  iVnd 

ElyperieiifieB  


Share  Areas  Of  Kxperlise  (Such  As 
Aft  Or  Com pu terg  j With  Each  Other 
Monitor  Membei^s*  Academic  Progress 
And  Ensure  They  Are  Achieving  


Provides  Inforinntion  About  What  3 
Late  Or  Absent  Member  Missed 

Help  Each  Other  Solve  Nunaeadeinic 
Problems 

Prepares  Meiubors  To  Take  Tests 
And  Go  Over  Questious  Missed 
Afterwards 

CoTtimunicate  Respect,  Liking,  And 
Confidence  In  One^s  Ability  Tn 
Manage  One^s  Challenges 

Attrition 


^Two  of  the  causes  of  dropping  out  of 
school  are  social  alienation  and 
academic  alienation. 

#Any  student  who  believes  that  "in  this 
schooi,  no  one  knows  me,  no  one  cares 
about  me,  no  one  wouid  miss  me  when 
i'm gone,  "is  at  the  risk  of  dropping  out. 
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Attrition 


#Base  groups  also  provide  a means 
of  fighting  a student's  inciination 
to  drop  out. 
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Changing  Students'  Attitudes 

About  Academic  Work 

1 

# There  are  several  general  principles, 
supported  by  research  (see Johnson  &d  F. 

J ohnson,  1997)  to  guide  faculty  efforts. 

1.  Attitudes  are  changed  in  groups,  not 
individual  by  individual. 

2.  Attitudes  are  changed  as  a result  of  small 
group  discussions  that  lead  to  public 
commitment  to  work  harder  in  school  and 
take  education  more  seriously. 
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Changing  Students'  Attitudes 
About  Academic  Work 


3.  Messages  from  individuals  who  care  about, 
and  are  committed  to,  the  students  are 
taken  more  seriously  than  messages  from 
indifferent  others. 


4.  Personally  tailor  appeals  to  value  education 
to  the  student.  General  messages  are  not 
nearly  as  effective  as  personal  messages. 
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Changing  Students'  Attitudes 
About  Academic  Work 


5.  Plan  for  the  long  term,  not  sudden 
conversions.  I nternationalization  of 
academic  values  will  take  years  of 
persuasion  by  caring  and  committed  peers 


6.  Support  from  caring  and  committed  peers 
essential  to  modifying  attitudes  and 
behaviors  and  maintaining  the  new 
attitudes  and  behaviors. 
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Meaning,  Purpose,  and 
Psychological  Health 


# Meaning  - is  primarily  created  from 
contributing  to  the  well  being  of  others  and 
the  common  good. 


■#The  significance  of  one's  actions  depend  on 
the  degree  to  which  one  balances  concern  for 
self  with  concern  for  others  and  the 
community  as  a whole. 
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Meaning,  Purpose,  and 
Psychological  Health 


# Young  adults  have  turned  away  from  careers 
of  public  service  to  careers  of  self-service. 


# Many  young  adults  have  a delusion  of 
individualism  believing  that: 

■ A.  they  are  separate  and  apart  from  other 
individuals  and  therefore, 

■ B.  others'  frustration,  unhappiness,  hunger, 
despair  and  misery  have  no  significant  bearing  on 
their  own  well-being 
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Conclusions 


#Base  Groups  - are  long-term 
heterogeneous  cooperative  leaning 
groups  with  stable  membership  whose 
primary  responsibilities  are  to  provide 
support,  encouragement,  and 
assistance  in  completing  assignments 
and  hold  each  other  accountable  for 
striving  to  learn. 
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Active  Learning 


Five  Basic  Eiements 


Chapter  Five 


David  W.  J ohnson 
Roger  T.  J ohnson 
Karl  A.  Smith 


(Interaction  Book  Company,  1998) 
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TYPES  OF  GROUPS 


your  undcmitinditig  of  different  ty^^s  ofg}'tnipH  by  rntiichins 
the  definitions  with  the  appreprinte  ^roup.  Check  your  answers  with  yout 


(.rtner  and  ij.xpiam  why  you  belicce  your  om^wers  tx*  hn  corryg^ 


1 Type  Of  Group  | 

Definition  | 

P^;iiudo  Group 

a,  A group  in  whicii  studonts  work  || 

together  lo  ncccjmpiish  shared  guale.  1 
Stud<3ntti  perteive  they  can  fetich  Oieir  1 
learning  goals  iT*and  only  il  the  other  I 
group  membert^  oleJo  reach  thoir  goiila.  1 

Traditional  hfejiniing^ 
Group 

b.  A group  vv'hose  membtra  hiive  been  || 

assigned  to  work  together  but  they  I 

havp  no  interetit  in  doing  so.  The  I 

aiructUTO  promotes  cOinp»?tihL/n  at  1 

clo-M*  c^uartera.  I 

Coaperutivc  Lcaminj^ 
Git>up 

d A group  that  all  the  criieria  lor  I 

being  a couppnjLive  group  and  1 

outpertornirt  all  reaaoiiable  1 

eNpecUitions,  given  Ito  mefinberBiuii.  1 

High-  Ferforraiiiice 
Cooperative  Learning 
Group 

c.  A group  who^yo  membera  agree  to  wtrrk  I 
toga  tiler,  but.  see  little  beiielii  from  1 

doing  so,  The  stiucture  pri)niot4ii?  I 

, individualistrc  work  N\*Uh  talking.  | 
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Basic  Elements  of  Cooperative  Learning 


Pseudo-  Learni no  Group 


■ Members  assigned  to  work  together  but  have 
no  interest  in  doing  do. 


■ Members  often  will: 

♦ Block  or  interfere  with  each  other 

♦ Communicate  and  coordinate  poorly 

♦ Mislead  or  confuse  each  other 

♦ Not  participate  and  seek  free  ride 

The  sum  of  the  whole  is  less  than  the  potential  of  the  individual  members 
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Basic  Elements  of  Cooperative  Learning 


^Traditional  Classroom  Learning  Group 


■ Members  accepted  that  they  are  to  work 
together,  but  see  little  benefit  from  doing  so. 

■ Member's: 

♦ Only  take  responsibility  for  themselves 

♦ Interact  primarily  to  share  information 

♦ Each  do  work  on  thei  r own 

♦ Accountable  as  individuals  not  as  team 
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Basic  Elements  of  Cooperative  Learning 


# Cooperative  Learning  Group 


■ A group  whose  members  are  committed  to  the  common 
purpose  of  maximizing  each  other's  learning 


■ Characteristics 


♦ Maximizing  aii  members'  iearning 

♦ Focus  both  on  group  and  individual  accountability 

♦ Members  do  real  work  together 

♦ Members  taught  social  skills 

♦ Groups  analyze  how  effectively  they  are  at  achieving  their  goals 
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Basic  Elements  of  Cooperative  Learning 


High-Performance  Coop  Learning  Groups^ 


■ Coop  learning  group  that  exceeds  all 
reasonable  expectations 


♦ Higher  level  of  commitment 

♦ Rare 

♦ Most  groups  never  achieve  this  level 
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Figure  5.1  Cooperative  Efforts 


ACHIEVEMENT 

PRODUCTIVITY 


»••••«  a:  • Xf 


PSYCHOLOGICAL 

HEALTH 


1 »T.  • 'VJ*. 
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Johnson,  Johnson,  & Smith  1998 


Figure  5.2  The  learning  Grour 


High-performing 


5 : 5 
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Johnson,  Johnson,  & Smith  (1998) 


Itable  1,1  Comparison  Of  Learning  Groups| 

'■ — il 

]| TRADITIONAL  LEARNING  GfOTUPS 

COOPERATrvH  LEAHMING  GROUPS  J 

|[L>y<  ivIficnberE  izhJt 

|i  frJUJvWJfihflity  tm]y  Pir  I'’npuf=r  ip  on 

■ individual  uertormarus  miy 

High  p(Hritlv«  Mumbt:!*?  1 

ji'e  rt  ipunaihle  for  *u’ti  mill  tach  ni1ii?r":5  1 

IwirmuK.  Fuci/i^  *s  on  lOlrti  pcTlurniunw: 

llndlviduul  only. 

Hoth  frru'ip  Ltul  ijidivWu*l  opcoujiLdljility 
Mcinbe.rR  .lold  5<ilf  oilviT  liMJnuritahIa  for 

tiich  mui'K.. 

Atyujnituflit-'i  vioth  litllt 

voinmltinent-to  «JMch  oUiorV  tyurjimR. 

Men»b«ra  pioinoli  twh  others  smicest:'.  Thcv 
<lrj  n.*nl  work  loge’.lrer.und  hvtn  Mod  .iiippcirt 
prtcli  or.hur  h fclTorl*i  to 

Tp4n»m»rk  skillt  ••iw.it'iim'url.  Landpr  la 
»ippijiTjti?(l  in  dirpct  TTuJmhtT^*  purti-mtui  ion 

Ti:arOWoi*H.nl;il|5  are  £^mpliiiL‘i2ocl  Mpmhort;  | 
an?  tauntiL  luul  uxpi.'Li.crt  in  u=e  ‘HXiai  •ikili'?, 

•Vli  mt^fohpr^  ahufF  ]fcaderi:hip 
re«pmwiblHLIri» 

Nu  group  pn»mjir-iii^\jrTlVF  c\nxlit>'  of  its 
1 wuiii-  Imliviilual  DiicumpInihnlrnW  are 
lirPvi'aillFd. 

CTrOiip  prtH!FaPEia  qiidllbi>’  rif  Wfir*k  and  hoW 
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Basic  Elements  of  Cooperative  Learning 


Forces  Hindering  Group  Performance 


■ Lack  of  group  maturity 

■ Uncritically  giving  one's  dominant  response 

■ Social  loafing  - hiding  in  the  crowd 

■ Free  riding 

■ Motivation  losses  due  to  perceived  inequity 

■ Group  thinking 

■ Lack  of  sufficient  heterogeneity 

■ Lack  of  teamwork  skills 

■ I nappropriate  group  size  juie  carborough,  2006 


Positive  I nterdependence:  we  i nstead  of  Me 


# Three  steps  to  structuring  positive  interdependence: 


■ Assigning  the  group  a ciear,  measurabie  task 

♦ Members  need  to  know  what  to  do 


■ structure  positive  goal  interdependence 

♦ All  members  scoring  above  a specified  criterion  when  tested 
individually 

♦ All  members  improve  their  performance  over  their  previous  scores 

♦ The  overall  group  score  being  above  a specified  criterion 

♦ One  product  successfully  completed  by  the  group 


Jule  Scarborough,  2006 


Positive  I nterdependence:  we  i nstead  of  Me 




■ Supplement  positive  goal  interdependence 
with  other  types  of  positive  interdependence: 


♦ Reward  interdependence 

♦ Resource  interdependence 

♦ Role  interdependence 

♦ Identify  interdependence 

♦ Environmental  interdependence 

♦ Fantasy  interdependence 

♦ Task  interdependence 

♦ Outside  enemy  interdependence  (see  p.5: 10) 

Jule  Scarborough,  2006 


I ndividual  Accountability/  Personal 

Responsibility 


^The  discipline  of  using  cooperative  groups 
inciudes  structuring  group  and  individuai 
accountability: 


■ Group  accountability:  The  performance  of  the 
overall  group  is  assessed  and  given  back  to  all 
group  members 


■ I ndividual  accountability:  The  performance  of 
each  individual  member  is  assessed,  the  results 
are  given  back  to  the  individual  and  the  group  to 

Jule  Scarborough,  2006 


I ndividual  Accountability/  Personal 

Responsibility 


The  purpose  of  cooperative  groups  is  to  make  eaci 
member  a Stronger  individual.  {note- reverse 


■ With  cooperative  groups: 

♦First,  Students  learn  knowledge,  skills, 
strategies,  or  procedures  in  a cooperative  group 


♦Then,  Students  apply  the  knowledge  or  perform 
the  skill,  strategy,  or  procedure  alone  to 
enhance  mastery  of  material 


Jule  Scarborough,  2006 


I ndividual  Accountability/  Personal 

Responsibility 


Positive  interdependence  and  accountability 


■ Personal  accountability 


♦ Contributing  his  or  her  efforts  to  accomplish  the  groups  goals 

♦ Helping  other  group  members  do  likewise 

♦ Do  what  ought  to  be  done 

■ Sharing 

■ Contribute 

■ Pulling  one's  weight 

■ ( attaining  competencies  individuaiiy,  but  at  higher  ievei  in  group) 


Jule  Scarborough,  2006 


Face-To-Face  Promotive  I nteraction 




^The  discipline  of  using  cooperative  groups  includes 
ensuring  that  group  members  meet  face- to- face 
to  work  together  to  complete  assignments  and 
promote  each  other's  success  ( note-  virtual) 


■ Schedule  time  for  the  group  to  meet 

■ Highlight  positive  interdependence  that  requires 
groups  to  work  together 


Encourage  promotive  interaction  among  group 
members  , , , , , 

Jule  Scarborough,  2006 


I nterpersonal  and  Small  Group  Skills 


^Teaching  group  members  the  small  group  and 


interpersonal  skills  they  need  to  work  effectively 
with  each  other 


■ Students  are  required  to  learn: 

♦ academic  subject  matter  (taskwork) 


♦ the  interpersonal  and  small  group  skills  required  to 
function  as  part  of  a group  (teamwork) 

No  Teamwork  Skills, No  Taskwork 

Jule  Scarborough,  2006 


Group  Processing 


#Group  processing 


■ Reflecting  on  a group  session  to: 

♦ Describe  what  members  actions  were  helpful  or 
unhelpful 

♦ Make  decisions  about  what  actions  to  continue  or 
change 


■ Purpose  is  to  clarify  and  improve  the 
effectiveness  of  the  members  to  achieve  the 
group  goals 


Jule  Scarborough,  2006 


Group  Processing 


# Five  steps  in  structuring  group  processing: 


■ Assess  the  quality  of  the  interaction  among  group 
members  as  they  work  to  maximize  each  other's 
learning. 

■ Give  each  learning  group  feedback  on  how  the  group 
does  its  work 

■ Groups  set  aoals  as  to  improve  their  effectiveness 

■ Process  how  effectively  the  whole  class  is  functional 

■ Conduct  small-group  and  whole-class  celebrations 


Jule  Scarborough,  2006 


Positive  I nterdependence  & 
I ntellectual  Conflict 


#The  greater  positive  interdependence, 

the  greater  the  likelihood  of  . . . 

intellectual  disagreement  and  conflict 

among  group  members 

**This  is  a good  thing! 


Jule  Scarborough,  2006 


Reducing  Problem  Behaviors 


'#When  students  first  start  working  in 
cooperative  learning  groups,  they  sometimes 
engage  in  unhelpful  behaviors. 

^Whenever  inappropriate  student  behavior 
occurs,  your  first  move  should  be  toward 

strengthening  the  perceived 
i nterdependence 


Jule  Scarborough,  2006 


Reducing  Problem  Behaviors 


❖ 


Student  Not  Participating  or  Bringing  Work  or 
Materials: 


Assign  student  role  essential  for  group  success 


Reward  group  if  all  members  achieve  up  to  criterion 
to  increase  peer  pressure  to  participate 


Jule  Scarborough,  2006 


Reducing  Problem  Behaviors 


^ A Student  is  Talking  About  Everything 
But  the  Assignment:  ~ 


Give  a reward  the  student  or  group  finds 
especially  attractive 

Structure  task  so  Steady  contributions 

are  required  for  group  success 

Jule  Scarborough,  2006 


Reducing  Problem  Behaviors 




# A Student  is  Working  Alone  and  I gnoring  the 
Group  Discussion: 

Limit  resources  in  the  group  (if  there  is  only 
one  pencil,  the  student  will  be  unable  to 
work  alone) 

J igsaw  materials  so  that  the  students  cannot 
complete  the  assignment  without  other 

members'  information  Jule  Scarborough,  2006 


Reducing  Problem  Behaviors 


A Student  is  Refusing  to  Let  Other  Members 
l^articipate: 


1 


igsaw  resources 


Assign  other  members  essential  roles  (leader, 
recorder,  etc.) 


Reward  on  group  basis  of  the  lowest  two  scores  by 
group  members 


Jule  Scarborough,  2006 


Research 


Different  types  of  research: 

1 . Experimental  Research  (controlled  experimentation) 

2.  Survey  Research 

3.  Qualitative  Field  Research 

4.  Unobtrusive  Research  (content  analysis,  analyze  existing 

statistics,  historical  comparative  analysis) 

5.  Evaluation  Research 

6.  - Others  - 
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Two  general  components/phases  of  experimental  research: 


1 . Setting  up  appropriate  designs/models  for  collecting  data. 

2.  Appropriate  statistical  analysis  of  the  data  to  address  the 
research  questions. 


Actually  the  second,  development  of  statistical  techniques, 
preceded  the  first,  consideration  of  experimental  designs. 

Campbell  and  Stanley  (1966)  quoted  from  McCall  (1923): 

“There  are  excellent  books  and  courses  of  instruction  dealing  with 
the  statistical  manipulation  of  experimental  data,  but  there  is  little 
help  to  be  found  on  the  methods  of  securing  adequate  and  proper 
data  to  which  to  apply  statistical  procedure.” 


Campbell  and  Stanley’s  separately  published  chapter. 
Experimental  and  Quasi-Experimental  Designs  for  Research, 
remains  one  of  the  most  complete  and  most  read  discussions  on 
experimental  designs  for  research  in  education. 
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The  basic  experimental  inference: 


We  want  to  infer  that  the  treatment  or  intervention,  X,  caused  or 
was  directly  related  to  the  observations  made  (the  data  that  were 
collected): 


X O 

We  want  X to  be  the  only  explanation  for  O.  But  there  are  often 
other  plausible  explanations,  other  variables,  that  can  explain  O. 

These  other  variables  are  considered  threats  to  the  basic 
experimental  inference  - threats  to  the  internal  validity  of  our 
experiment. 

Our  research  is  weakened  to  the  extent  that  the  basic  experimental 
inference  is  threatened  by  alternative  explanations. 


Campbell  and  Stanley  consider  several  experimental  designs  and 
discuss  the  various  threats  to  the  validity  of  those  designs. 
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Design  1.  The  One-Shot  Case  Study 


X O 

This  design  suffers  from  many  weaknesses  (threats  to  its  validity). 
Some  will  be  discussed  as  we  look  at  a few  other  designs. 

But  the  primary  weakness  of  this  design  is  the  absence  of  any 
comparative  data. 


Campbell  and  Stanley: 

“. . . such  studies  have  such  a total  absence  of  control  as  to  be  of 
almost  no  scientific  value.  The  design  is  introduced  here  as  a 
minimum  reference  point.” 

“Basic  to  scientific  evidence  ...  is  the  process  of  comparison,  of 
recording  differences,  or  of  contrast.” 

In  this  design  the  inference  of  treatment  effectiveness  is  based  on  a 
comparison  “. . .with  other  events  casually  observed  and 
remembered,  . . .general  expectations  of  what  the  data  would  have 
been  had  the  X not  occurred,  etc.,  . . .and  in  such  instances  involve 
the  error  of  misplaced  precision” 

In  other  words,  we  just  kind  of  ‘feel’  the  treatment  was  effective 
because  the  data  appear  acceptable.  This  is  not  scientific. 
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Design  2.  The  One-Group  Pretest-Posttest  Design 


0X0 

A little  better  than  design  1 (there  is  a pretest  for  comparison  to  the 
posttest  scores),  but  the  design  suffers  from  several  weaknesses  - 
threats  to  its  validity. 


History.  Events  that  occur  between  the  pretest  and  posttest,  in 
addition  to  the  treatment  X,  that  could  effect  the  posttest  scores. 

Maturation.  Changes  in  subjects,  in  addition  to  the  treatment  X, 
that  could  cause  changes  in  posttest  scores.  These  are  “a  function 
of  the  passage  of  time  per  se  (not  specific  to  particular  events), 
including  growing  older,  growing  hungrier,  growing  more  tired, 
and  the  like.” 

Testing.  Actually  taking  the  pretest  may  effect  scores  on  the 
posttest  beyond  the  effects  of  the  treatment  X. 

Instrumentation.  The  pretest  and  posttest  may  be  different 
instruments  - they  shouldn’t  be.  “. . . [CJhanges  in  the  calibration 
of  a measuring  instrument  or  changes  in  the  observers  or  scorers 
used  may  produce  changes  in  the  obtained  measurements.” 

Regression.  Can  occur  when  subjects  have  extreme  (either  high  or 
low)  pretest  scores.  By  the  phenomenon  of  statistical  regression 
(or  regression  toward  the  mean)  at  the  posttest,  extreme  scorers 
tend  to  score  ‘away’  from  their  extreme  and  toward  the  mean. 
Posttest  scores  may  have  nothing  to  do  with  the  treatment  X. 
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Design  3.  Static-Group  Comparison 


.X...  O. 

o 

Two  groups;  one  group  received  the  treatment,  the  other  group  did 
not.  The  scores  from  the  two  groups  can  be  compared. 

The  problem  is  that  these  are  usually  existing  (static)  groups;  a 
group  that  happened  to  receive  the  treatment  is  compared  to  a 
group  that  did  not.  But  subjects  were  not  explicitly  (and  randomly) 
assigned  to  groups. 

This  design  is  not  weakened  by  the  five  previous  threats, 
history,  maturation,  testing,  instrumentation,  and  regression, 
because  these  variables  threaten  the  change  observed  between  the 
pretest  and  posttest;  this  model  has  only  one  testing  time. 

But  this  model  can  be  weakened  by  other  threats,  such  as: 

Selection.  One  group  of  subjects  may  be  qualitatively  different 
from  the  other  group,  at  least  in  regards  to  the  dependent  variable 
(whatever  the  test  is  measuring). 

Mortality.  Differential  loss  of  subjects  related  to  group 
membership  or  group  dynamics  or  other  group- specific  factors. 
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Design  6.  Posttest-Only  Control  Group  Design 


R X O 

R O 

The  difference  between  this  design  and  design  3,  the  static-group 
comparison,  is  in  the  random  assignment  (R)  of  subjects  to 
experimental  and  control  groups. 

This  is  a very  acceptable  design.  Like  design  3,  it  is  not  weakened 
by  history,  maturation,  testing,  instrumentation,  and  regression 
because  this  is  not  a pretest-posttest  design. 

In  addition,  Selection  is  not  a threat  because  random  assignment  to 
groups  assures  there  is  no  bias  in  group  membership.  Differential 
experimental  Mortality  is  also  not  a threat  because,  again  due  to 
random  assignment,  any  factors  resulting  in  the  loss  of  subjects 
should  affect  both  groups  equally. 
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Design  4.  Pretest-Posttest  Control  Group  Design 


R O X O 

R O O 


This  design  differs  from  design  6 by  the  inclusion  of  pretesting. 

It  has  the  advantage  of  design  6 in  not  being  susceptible  to 
selection  and  mortality  (because  of  randomization). 

The  design  appears  to  be  susceptible  to  history,  maturation, 
testing,  instrumentation,  and  regression.  These  variables  can,  in 
fact,  effect  the  pretest-posttest  comparisons  in  the  two  groups. 

But,  because  there  are  two  groups  AND  subjects  were  randomly 
assigned  to  each  group,  we  can  assume  that  the  effects  of  these 
variables  are  equalized  across  both  groups. 

So,  history,  maturation,  testing,  instrumentation,  and  regression 
are  not  avoided  or  eliminated  - they  are  controlled. 

This  is  probably  the  most  used  experimental  design.  Sometimes 
called  the  classical  experimental  design. 

Of  the  sixteen  designs  discussed  by  Campbell  and  Stanley,  designs 
4,  5,  and  6,  are  “the  most  strongly  recommended  designs  of  this 
presentation.” 
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Design  5.  Solomon  Four-Group  Design 


R O X O 

R O O 

R X O 

R O 

The  basic  appearance  of  this  design  is  the  combining  of  the  two 
groups  from  design  4 with  the  two  groups  from  design  6. 

Designs  4,  5,  and  6,  all  control  the  major  threats  to  internal 
validity.  Designs  5 and  6 have  the  added  advantage  of  controlling 
one  of  the  threats  to  external  validity. 

Internal  validity  relates  to  the  extent  to  which  change  can  be 
attributed  to  the  treatment  and  not  to  other  variables  (like  history, 
maturation,  etc.) 

External  validity  relates  to  the  appropriateness  of  generalizing 
experimental  results  to  other  populations,  settings,  treatment 
variables,  and  measurement  variables. 
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The  Basic  Logic  of  Inferential  Statistical  Methods 


Often,  in  the  experimental  designs  on  the  previous  pages,  we  wish 
to  compare  the  means  of  scores  from  two  groups:  experimental  vs. 
control,  or  pretest  vs.  posttest. 

Consider  and  ^2  • 

[In  statistics  the  symbol  ji  is  used  to  represent  the  true  mean  of  a 

group  and  X is  the  computed  mean  of  the  actual  data  from  the 
group.] 

The  logic  of  statistical  inference  requires  us  to  specify  a null 
hypothesis  which  states  that  the  two  true  means  are  equal  (even 
though  we  generally  hope  or  expect  that  they  are  unequal): 

The  statistical  test  will  then  compare  (test)  this  null  hypothesis 
against  a more  general  alternative  hypothesis: 

[or//i<//2  or 

Of  course,  the  actual  computed  means  will  likely  be  unequal. 

Let’s  assume  that  the  mean  for  the  experimental  group  is  10  points 

greater  than  the  mean  for  the  control  group  (^2“^i  = 10).  But  is 
this  10-point  difference  big  enough  for  us  to  infer  that  the 
experimental  treatment  is  really  effective? 

That  is  the  basic  question:  how  big  of  a difference  is  big  enough? 

Inferential  statistical  procedures  will  help  answer  that  question. 
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The  process  involves  probabilities,  normal  curves,  and  input  from 
the  researcher  and,  as  a result  of  the  statistical  test,  we  could 
determine  that  a mean  difference  of  7.34  points  (or  whatever)  is 
sufficient  for  us  to  judge  that  the  treatment  is  indeed  effective  in 
changing  scores. 

Based  on  the  statistical  test  we  will  either  retain  the  null  hypothesis 
or  reject  it  - that’s  all  the  statistical  procedure  allows  us  to  do. 

If  we  reject  it,  we  are  supporting  the  alternative  hypothesis  - for 
this  example  we  are  inferring  that  the  treatment  is  effective. 

If  we  retain  the  null,  we  are  not  supporting  the  alternative  - we  are 
inferring  that  the  treatment  is  not  effective. 

In  either  case,  there  is  a small  probability  that  we  are  wrong.  This 
means  that  inferential  statistical  procedures  do  not  allow  us  to 
conclusively  prove  that  the  treatment  is  effective  or  is  not 
effective. 
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Study  1 -Traditional  Assessment  With  Performance  Assessment 


This  study  is  designed  to  examine  two  questions: 

1.  Does  a performance  test  administered  in  conjunction  with  a traditional 
cognitive  test  result  in  increased  learning  beyond  the  traditional  test 
alone  as  indicated  by  the  traditional  cognitive  test? 

2.  Does  a performance  test  administered  in  conjunction  with  a traditional 
cognitive  test  result  in  increased  knowledge  retention  beyond  the 
traditional  test  alone  as  indicated  by  a final  exam? 

The  basic  design  for  this  study  is  presented  in  the  table  below. 


Study  1:  Traditional  Assessment  With  Performance  Assessment 

Treatment 

Posttest  1 

Posttest  2 

Experimental 

Group 

Instruction 

Performance 
Test  Related  to 
Traditional  Test 

Traditional 

Test 

-> 

Traditional  Final 

Control 

Group 

Instruction 

Performance 
Test  Not 
Related  to 
Traditional  Test 
(Placebo) 

Traditional 

Test 

-> 

Traditional  Final 
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Study  2 - Performance  Assessment  and  Traditional  Assessment 
Administered  in  Different  Order 


This  study  is  designed  to  examine  the  following  questions: 

1 . Does  a performance  test  administered  in  conjunction  with  a traditional 
cognitive  test  result  in  increased  learning  beyond  the  traditional  test 
alone  as  indicated  by  the  traditional  cognitive  test? 

2.  Does  the  order  of  administration  of  a performance  test  and  a traditional 
test  affect  knowledge  retention  as  indicated  by  a final  exam? 

The  basic  design  for  this  study  is  presented  in  the  table  below. 


Study  2:  Performance  Assessment  and  Traditional  Assessment 
Administered  in  Different  Order 

Group  1 

Instructio 

n 

Performa 
nee  Test 

Traditional 

Test 

Final 

Group  2 

Instructio 

n 

Traditional 

Test 

Performan 
ce  Test 

Final 
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Study  3 - Passive  Learning  vs.  Active  Learning 


This  study  is  designed  to  examine  the  following  questions: 

1 . Does  passive  learning  vs.  active  learning  result  in  differential 
knowledge  gains  as  indicated  by  a traditional  cognitive  test? 

2.  Does  passive  learning  vs.  active  learning  result  in  differential 
knowledge  retention  as  indicated  by  a final  exam? 

The  design  for  this  study  is  presented  in  the  table  below. 


Study  3:  Passive  Learning  vs.  Active  Learning 


Treatment 

Posttest  1 

Posttest  2 

Passive 

Learning 

Group 

Passive 

Learning 

Traditional  Test 

Final 

Active 

Learning 

Group 

Active  Learning 

Traditional  Test 

Final 

Treatment 

Group 

Content  Area 
I 

Content  Area 
II 

Content  Area 
III 

Content  Area 
IV 

1 

Passive 

Active 

Passive 

Active 

2 

Active 

Passive 

Active 

Passive 
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Study  4 - Individual  Learning  vs.  Cooperative  Learning 


This  study  is  designed  to  examine  the  following  questions: 

1.  Does  individual  learning  vs.  cooperative  learning  result  in  differential 
knowledge  gains  as  indicated  by  a traditional  cognitive  test? 

2.  Does  individual  learning  vs.  cooperative  learning  result  in  differential 
knowledge  retention  as  indicated  by  a final  exam? 

The  design  for  this  study  is  presented  in  the  table  below. 


Study  4:  Individual  Learning  vs.  Cooperative  Learning 


Treatment 

Posttest  1 

Posttest  2 

Individual 

Learning 

Group 

Individual 

Learning 

Traditional  Test 

Final 

Cooperative 

Learning 

Group 

Cooperative 

Learning 

Traditional  Test 

Final 

Treatment 

Group 

Content  Area 
I 

Content  Area 
II 

Content  Area 
III 

Content  Area 
IV 

1 

Individual 

Cooperative 

Individual 

Cooperative 

2 

Cooperative 

Individual 

Cooperative 

Individual 
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